2.5 Bootstrapping and Random Forest
[video] /api/lesson-media/videos/9b417aa2-5e9f-4ae3-a9db-2a3b7f03f372-1794615016446-b3ee9ef22bb2858b.mp4
[markdown] content available only on the platform
[markdown] content available only on the platform
Flashcards · 2.5 Bootstrapping and Random Forest
Ensemble method — A strategy that combines many individual models to produce a better collective prediction than any one of them.
Random Forest combines many decision trees.
Bootstrapping — Creating new samples by drawing randomly with replacement from the original dataset.
From A..G, the sample D, E, F, D is valid.
Sampling with replacement — Each picked item is returned to the pool, so it can be picked again in the same sample.
B, A, B, B contains B three times.
Random Forest — An ensemble of decision trees, each trained on a different bootstrap sample with random feature choices.
100 trees vote on whether to approve a loan.
Majority voting — How a classification forest combines trees: the class predicted by the most trees wins.
4 trees say Reject, 3 say Accept -> Reject.
Averaging (regression forest) — How a regression forest combines trees: it outputs the mean of all the trees' predictions.
Trees predict 100, 120, 110 -> forest predicts 110.
Overfitting — Learning the training data too specifically, including noise, so performance on new data is poor.
A deep tree scores 100% on train but much lower on test.
RandomForestClassifier — The scikit-learn class for Random Forest classification; RandomForestRegressor is the regression version.
RandomForestClassifier(n_estimators=100, random_state=100)
