Decision Tree Learning
•Decision Tree is a Supervised learning technique that
can be used for both classification and Regression
problems, but mostly it is preferred for solving
Classification problems.
•It is a tree-structured classifier, where internal nodes
represent the features of a dataset, branches represent
the decision rules and each leaf node represents the
outcome.
•In a Decision tree, there are two nodes, which are the
Decision Node and Leaf Node.
•Decision nodes are used to make any decision and have
multiple branches, whereas Leaf nodes are the output of
those decisions and do not contain any further branches.
•The decisions or the test are performed on the basis of
features of the given dataset.
•It is a graphical representation for getting all the possible
solutions to a problem/decision based on given
conditions.
•It is called a decision tree because, similar to a tree, it
starts with the root node, which expands on further
branches and constructs a tree-like structure.
•In order to build a tree, we use the CART algorithm, which
stands for Classification and Regression Tree algorithm.
•A decision tree simply asks a question, and based on the
answer (Yes/No), it further split the tree into subtrees.
Why use Decision Trees?
There are various algorithms in Machine learning, so
choosing the best algorithm for the given dataset and
problem is the main point to remember while creating a
machine learning model.
Below are the two reasons for using the Decision tree:
•Decision Trees usually mimic human thinking ability while
making a decision, so it is easy to understand.
•The logic behind the decision tree can be easily understood
because it shows a tree-like structure.
Decision Tree Terminologies
Root Node: Root node is from where the decision tree starts.
It represents the entire dataset, which further gets divided
into two or more homogeneous sets.
Leaf Node: Leaf nodes are the final output node, and the tree
cannot be segregated further after getting a leaf node.
Splitting: Splitting is the process of dividing the decision
node/root node into sub-nodes according to the given
conditions.
Branch/Sub Tree: A tree formed by splitting the tree.
Pruning: Pruning is the process of removing the unwanted
branches from the tree.
Parent/Child node: The root node of the tree is called the
parent node, and other nodes are called the child nodes.
How does the Decision Tree algorithm Work?
Step-1: Begin the tree with the root node, says S, which
contains the complete dataset.
Step-2: Find the best attribute in the dataset using Attribute
Selection Measure (ASM).
Step-3: Divide the S into subsets that contains possible values
for the best attributes.
Step-4: Generate the decision tree node, which contains the
best attribute.
Step-5: Recursively make new decision trees using the
subsets of the dataset created in step -3. Continue this
process until a stage is reached where you cannot further
classify the nodes and called the final node as a leaf node.
Example
A decision tree represents data using if-then-else rules.
Weather Temperature Humidity Wind Carry Umbrella
Sunny Hot High Weak No
Rainy Mild Normal Strong Yes
Overcast Cool High Weak Yes
Rainy Mild High Weak Yes
Weather?
/ | \
Sunny Overcast Rainy
/ \ \
No Yes Humidity?
/ \
High Normal
Yes Yes
Algorithm: ID3
Steps:
1. Start with all data at the root.
2. Choose the best attribute to split on (using a heuristic
like Information Gain).
3. Split the dataset into subsets by that attribute.
4. Repeat recursively on each subset until:
o All records in a node have the same label.
o No more attributes to split.
o Dataset is empty
Weather Temperature Humidity Wind Carry Umbrella
Sunny Hot High Weak No
Sunny Hot High Strong No
Overcast Mild High Weak Yes
Rainy Cool Normal Weak Yes
Rainy Cool Normal Strong No
Overcast Cool Normal Strong Yes
Sunny Mild High Weak No
Rainy Mild High Weak Yes
Sunny Mild Normal Strong No
