You are viewing a public preview
Sign in to save your progress, do assignments, join discussions and get feedback.
After this unit you will: name the types of nodes in a tree correctly and read the Yes/No count table when splitting on each attribute.
loan_approved, and four attributes: age, income, is_student, credit_rating.Split by age | Yes | No |
|---|---|---|
| young | 2 | 3 |
| middle | 4 | 0 |
| senior | 3 | 2 |
By the end of this lesson you can:
loan_approvedKey idea: A decision tree is a flowchart of yes/no questions learned from data; the algorithm chooses which question to ask first by checking which feature splits the data into the purest groups.
The lecture introduces decision trees with a job application example. It is probably the most intuitive algorithm in the course because it mimics how humans make decisions.
So the path "bachelor degree = Yes, Excellent degree = No, IELTS = Yes" visits the root node, then two decision nodes, and ends at the leaf Accept. A decision tree is just a series of nested questions, each narrowing down the data until a leaf is reached.
In scikit-learn the algorithm is DecisionTreeClassifier. The real strength is that the tree is not built from arbitrary questions: it learns from the training data which question to ask and in which order.
To see how the tree learns, the lecture uses a classic teaching dataset (very similar versions appear in many textbooks). It has 14 rows with the columns ID, age, income, student, credit rating and loan_approved.
| Feature | Type | Values |
|---|---|---|
| age | ordinal | young, middle, senior |
| income | ordinal | high, medium, low |
student (is_student) | Boolean | yes, no |
| credit rating | ordinal | fair, excellent |
The target loan_approved is yes or no, so this is a binary classification problem. The key question is: which of the four features should the tree ask about first, as the root node?
For each feature we split the 14 applicants into groups and count the Yes and No outcomes in each group. Splitting on age gives:
| Split by age | Yes | No |
|---|---|---|
| young | 2 | 3 |
| middle | 4 | 0 |
| senior | 3 | 2 |
A group is pure when it contains mostly or only one outcome. The middle-aged group (4 Yes, 0 No) is perfectly pure: every middle-aged applicant got the loan. That is a strong, clean signal. The young group (2 Yes, 3 No) is a mixed group and much less informative on its own.
Notice that whichever feature you split on, the counts across its branches always add up to the same 14 applicants; only the way they are grouped changes. To compare features objectively, the algorithm needs a number that measures purity. That number is the Gini index (Gini impurity), the topic of the next lesson.
A bank officer wants to understand the age split before any formula is used.
In code, scikit-learn needs numbers, so categories such as young/middle/senior must be encoded before training:
from sklearn.tree import DecisionTreeClassifier
model = DecisionTreeClassifier(random_state=100)
model.fit(x_train, y_train) # x_train: encoded age, income, student, credit rating
print(model.predict(x_test[:5]))
loan_approved is the target the model predicts; it is not one of the four candidate features.loan_approved.Flashcards · 2.3 Decision Trees and the Loan Approval Problem
9 wordsA model that predicts by asking a learned sequence of yes/no questions, like a flowchart, until it reaches a final answer.
The first question at the top of the tree, where every prediction starts.
A node that asks a condition and splits the data further; the root is a special decision node.
An endpoint of the tree that asks no more questions and outputs the final decision.
A branch whose samples all share one label, so no further splitting is needed.
Dividing the samples at a node into groups according to the values of one feature.
The binary target column of the loan dataset that the tree learns to predict.
A categorical feature whose values have a natural order.
The scikit-learn class that learns a decision tree for classification problems.