Back to the lessonPreparing the printout…

Fundamental of Machine Learning · Module 2: Core Machine Learning Algorithms

2.4 Gini impurity

[video] /api/lesson-media/videos/9b417aa2-5e9f-4ae3-a9db-2a3b7f03f372-1794614948940-03a62b81b350b79c.mp4

[markdown] content available only on the platform

[markdown] content available only on the platform

Flashcards · 2.4 Gini impurity

Gini impurity — A measure of how mixed the labels in a group are: 1 minus the sum of squared class proportions.

2 Yes, 3 No: 1 - (0.4^2 + 0.6^2) = 0.48

Gini = 0 — The group is perfectly pure: every sample has the same label, so no further split is needed.

age = middle: 4 Yes, 0 No.

Maximum Gini (two classes) — The highest Gini possible with two classes is 0.5, reached by an exact 50/50 mix.

5 Yes, 5 No gives Gini = 0.5.

Weighted Gini — The Gini of a feature: the average of its branches' Gini values, each weighted by its share of the samples.

(5/14)*0.48 + (4/14)*0 + (5/14)*0.48 = 0.343

Branch weight — The proportion of samples that fall into a branch, used to weight that branch's Gini.

5 of 14 applicants are young: weight 5/14.

Best split rule — The tree chooses the feature with the lowest weighted Gini, because it gives the purest groups.

Compare Gini(age), Gini(income), Gini(student), Gini(credit rating).

Recursive splitting — Repeating the best-split search at every new decision node on its own subgroup of data.

After age, split the young branch by student status.

max_depth — A stopping condition that limits how many levels of questions the tree may grow.

DecisionTreeClassifier(max_depth=4)

Fundamental of Machine Learning · 2.4 Gini impurity · printed on November 21, 2026