Back to the lessonPreparing the printout…

Fundamental of Machine Learning · Module 2: Core Machine Learning Algorithms

2.5 Bootstrapping and Random Forest

[video] /api/lesson-media/videos/9b417aa2-5e9f-4ae3-a9db-2a3b7f03f372-1794615016446-b3ee9ef22bb2858b.mp4

[markdown] content available only on the platform

[markdown] content available only on the platform

Flashcards · 2.5 Bootstrapping and Random Forest

Ensemble method — A strategy that combines many individual models to produce a better collective prediction than any one of them.

Random Forest combines many decision trees.

Bootstrapping — Creating new samples by drawing randomly with replacement from the original dataset.

From A..G, the sample D, E, F, D is valid.

Sampling with replacement — Each picked item is returned to the pool, so it can be picked again in the same sample.

B, A, B, B contains B three times.

Random Forest — An ensemble of decision trees, each trained on a different bootstrap sample with random feature choices.

100 trees vote on whether to approve a loan.

Majority voting — How a classification forest combines trees: the class predicted by the most trees wins.

4 trees say Reject, 3 say Accept -> Reject.

Averaging (regression forest) — How a regression forest combines trees: it outputs the mean of all the trees' predictions.

Trees predict 100, 120, 110 -> forest predicts 110.

Overfitting — Learning the training data too specifically, including noise, so performance on new data is poor.

A deep tree scores 100% on train but much lower on test.

RandomForestClassifier — The scikit-learn class for Random Forest classification; RandomForestRegressor is the regression version.

RandomForestClassifier(n_estimators=100, random_state=100)

Fundamental of Machine Learning · 2.5 Bootstrapping and Random Forest · printed on November 21, 2026