2.4 Gini impurity
[video] /api/lesson-media/videos/9b417aa2-5e9f-4ae3-a9db-2a3b7f03f372-1794614948940-03a62b81b350b79c.mp4
[markdown] content available only on the platform
[markdown] content available only on the platform
Flashcards · 2.4 Gini impurity
Gini impurity — A measure of how mixed the labels in a group are: 1 minus the sum of squared class proportions.
2 Yes, 3 No: 1 - (0.4^2 + 0.6^2) = 0.48
Gini = 0 — The group is perfectly pure: every sample has the same label, so no further split is needed.
age = middle: 4 Yes, 0 No.
Maximum Gini (two classes) — The highest Gini possible with two classes is 0.5, reached by an exact 50/50 mix.
5 Yes, 5 No gives Gini = 0.5.
Weighted Gini — The Gini of a feature: the average of its branches' Gini values, each weighted by its share of the samples.
(5/14)*0.48 + (4/14)*0 + (5/14)*0.48 = 0.343
Branch weight — The proportion of samples that fall into a branch, used to weight that branch's Gini.
5 of 14 applicants are young: weight 5/14.
Best split rule — The tree chooses the feature with the lowest weighted Gini, because it gives the purest groups.
Compare Gini(age), Gini(income), Gini(student), Gini(credit rating).
Recursive splitting — Repeating the best-split search at every new decision node on its own subgroup of data.
After age, split the young branch by student status.
max_depth — A stopping condition that limits how many levels of questions the tree may grow.
DecisionTreeClassifier(max_depth=4)
