Back to the lessonPreparing the printout…

Fundamental of Machine Learning · Module 1: Data Preparation and Model Foundations

Supervised Learning Foundations and the Bias-Variance Tradeoff

[markdown] content available only on the platform

[markdown] content available only on the platform

Flashcards · Supervised Learning Foundations and the Bias-Variance Tradeoff

Supervised learning — Learning a mapping from inputs to outputs using training samples that each carry the correct label.

Learning house prices from past sales.

Bias — Error caused by a model being too simple to capture the real pattern.

A straight line fitted to a curved relationship.

Variance — Error caused by a model being so flexible that it changes a lot with the training data and fits noise.

A very deep tree that memorises every point.

Underfitting — High-bias situation where the model has high error on both training and test data.

Train MSE 0.40, test MSE 0.42.

Overfitting — High-variance situation where training error is very low but test error is high.

99% train accuracy, 62% test accuracy.

Bias-variance tradeoff — As complexity rises, bias falls and variance rises; the best model minimises total error on new data.

Choosing max_depth=3 over 1 or unlimited.

max_depth — Decision tree setting that limits how many levels of splits the tree can grow, controlling complexity.

DecisionTreeRegressor(max_depth=3)

Regularization — A technique that penalises model complexity to reduce overfitting.

Adding a penalty on large coefficients.

Train-test gap — The difference between training and test error; a large gap signals overfitting.

Train error 0.00 vs test error 0.30.

Fundamental of Machine Learning · Supervised Learning Foundations and the Bias-Variance Tradeoff · printed on November 20, 2026