Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 2: Core Machine Learning Algorithms

2.3 Decision Trees and the Loan Approval Problem

Print / Save as PDF

2.3 Decision Trees and the Loan Approval Problem

After this unit you will: name the types of nodes in a tree correctly and read the Yes/No count table when splitting on each attribute.

Key points

  • Root node: the top node. Decision node: a node that asks about a condition. Leaf node: a node holding the final conclusion (Accept / Reject).
  • Going from the root down to a leaf, following the Yes/No answers, yields a prediction.
  • The loan approval example has 14 samples, the label loan_approved, and four attributes: age, income, is_student, credit_rating.
  • For each attribute, we count the Yes/No samples in each branch to compare the “purity” of the split.
Split by ageYesNo
young23
middle40
senior32

Study notes

By the end of this lesson you can:

  • Name the three kinds of nodes in a decision tree: root node, decision node and leaf node
  • Follow a path from the root to a leaf to make a prediction
  • Describe the bank loan approval dataset: 14 applicants, four features and the label loan_approved
  • Read a Yes/No count table for a split and judge which branches are pure and which are mixed
  • Explain why the tree needs a formal purity measure (Gini) to choose its first question

Key idea: A decision tree is a flowchart of yes/no questions learned from data; the algorithm chooses which question to ask first by checking which feature splits the data into the purest groups.

A flowchart of yes/no questions

The lecture introduces decision trees with a job application example. It is probably the most intuitive algorithm in the course because it mimics how humans make decisions.

  • Root node: the very first question at the top. Here: does the applicant have a bachelor degree? Every prediction starts here.
  • Decision node: any question further down that splits the data again, such as "Excellent degree?" or "Experience?". The root node is itself a special decision node.
  • Leaf node: an endpoint where no more questions are asked and the tree outputs its decision: Accept or Reject.

Walking through the job application tree

  1. Root: bachelor degree? If yes, ask: is it an excellent degree?
    • Excellent degree = yes: Accept.
    • Excellent degree = no: ask one more question, does the applicant have an IELTS certificate? Yes: Accept; no: Reject.
  2. Root answer no (no bachelor degree): ask about experience. Has experience: Accept; no experience: Reject.

So the path "bachelor degree = Yes, Excellent degree = No, IELTS = Yes" visits the root node, then two decision nodes, and ends at the leaf Accept. A decision tree is just a series of nested questions, each narrowing down the data until a leaf is reached.

In scikit-learn the algorithm is DecisionTreeClassifier. The real strength is that the tree is not built from arbitrary questions: it learns from the training data which question to ask and in which order.

The bank loan approval dataset

To see how the tree learns, the lecture uses a classic teaching dataset (very similar versions appear in many textbooks). It has 14 rows with the columns ID, age, income, student, credit rating and loan_approved.

FeatureTypeValues
ageordinalyoung, middle, senior
incomeordinalhigh, medium, low
student (is_student)Booleanyes, no
credit ratingordinalfair, excellent

The target loan_approved is yes or no, so this is a binary classification problem. The key question is: which of the four features should the tree ask about first, as the root node?

Measuring how pure a split is

For each feature we split the 14 applicants into groups and count the Yes and No outcomes in each group. Splitting on age gives:

Split by ageYesNo
young23
middle40
senior32

A group is pure when it contains mostly or only one outcome. The middle-aged group (4 Yes, 0 No) is perfectly pure: every middle-aged applicant got the loan. That is a strong, clean signal. The young group (2 Yes, 3 No) is a mixed group and much less informative on its own.

Notice that whichever feature you split on, the counts across its branches always add up to the same 14 applicants; only the way they are grouped changes. To compare features objectively, the algorithm needs a number that measures purity. That number is the Gini index (Gini impurity), the topic of the next lesson.

Worked example

A bank officer wants to understand the age split before any formula is used.

  1. Count totals per branch: young 2 + 3 = 5, middle 4 + 0 = 4, senior 3 + 2 = 5. Total 5 + 4 + 5 = 14, which matches the dataset.
  2. Find pure branches: only "middle" has a single label (all Yes). If the tree splits on age, the middle branch can become a leaf, Accept, straight away.
  3. Find mixed branches: young (2 vs 3) and senior (3 vs 2) are close to 50/50, so the tree must keep asking questions on those branches, for example about student status or credit rating.
  4. Conclusion: age gives one perfectly pure branch and two mixed ones. Whether it is the best root depends on how the other three features compare, which needs a formal score.

In code, scikit-learn needs numbers, so categories such as young/middle/senior must be encoded before training:

from sklearn.tree import DecisionTreeClassifier

model = DecisionTreeClassifier(random_state=100)
model.fit(x_train, y_train)   # x_train: encoded age, income, student, credit rating
print(model.predict(x_test[:5]))

Common mistakes

  • Confusing node types. The root is the top question; leaves hold the final answer; everything in between is a decision node.
  • Thinking the tree's questions are written by hand. The algorithm learns which question to ask, and in which order, from the training data.
  • Calling a branch pure because it has more Yes than No. Pure means one single label (like 4 Yes, 0 No); 3 Yes and 2 No is still mixed.
  • Forgetting the label column. loan_approved is the target the model predicts; it is not one of the four candidate features.

Summary

  • A decision tree is a flowchart of nested yes/no questions ending in leaf decisions.
  • Root node = first question, decision nodes = later questions, leaf nodes = final outcomes.
  • The loan dataset has 14 applicants, four features (age, income, student, credit rating) and the binary label loan_approved.
  • Splitting on a feature produces groups whose Yes/No counts show how pure each branch is.
  • The middle age group (4 Yes, 0 No) is perfectly pure; the tree needs Gini impurity to compare all splits fairly.

Flashcards · 2.3 Decision Trees and the Loan Approval Problem

9 words
WordMeaningExample
  • Decision tree

    A model that predicts by asking a learned sequence of yes/no questions, like a flowchart, until it reaches a final answer.

    Bachelor degree? -> Excellent degree? -> Accept.

  • Root node

    The first question at the top of the tree, where every prediction starts.

    "Does the applicant have a bachelor degree?"

  • Decision node

    A node that asks a condition and splits the data further; the root is a special decision node.

    "Excellent degree?" in the middle of the tree.

  • Leaf node

    An endpoint of the tree that asks no more questions and outputs the final decision.

    Accept or Reject.

  • Pure branch

    A branch whose samples all share one label, so no further splitting is needed.

    age = middle: 4 Yes, 0 No.

  • Split

    Dividing the samples at a node into groups according to the values of one feature.

    Splitting 14 applicants by age into young, middle, senior.

  • loan_approved

    The binary target column of the loan dataset that the tree learns to predict.

    Values: yes or no.

  • Ordinal feature

    A categorical feature whose values have a natural order.

    income: low < medium < high.

  • DecisionTreeClassifier

    The scikit-learn class that learns a decision tree for classification problems.

    DecisionTreeClassifier(random_state=100).fit(x_train, y_train)