Supervised Learning Foundations and the Bias-Variance Tradeoff
[markdown] content available only on the platform
[markdown] content available only on the platform
Flashcards · Supervised Learning Foundations and the Bias-Variance Tradeoff
Supervised learning — Learning a mapping from inputs to outputs using training samples that each carry the correct label.
Learning house prices from past sales.
Bias — Error caused by a model being too simple to capture the real pattern.
A straight line fitted to a curved relationship.
Variance — Error caused by a model being so flexible that it changes a lot with the training data and fits noise.
A very deep tree that memorises every point.
Underfitting — High-bias situation where the model has high error on both training and test data.
Train MSE 0.40, test MSE 0.42.
Overfitting — High-variance situation where training error is very low but test error is high.
99% train accuracy, 62% test accuracy.
Bias-variance tradeoff — As complexity rises, bias falls and variance rises; the best model minimises total error on new data.
Choosing max_depth=3 over 1 or unlimited.
max_depth — Decision tree setting that limits how many levels of splits the tree can grow, controlling complexity.
DecisionTreeRegressor(max_depth=3)
Regularization — A technique that penalises model complexity to reduce overfitting.
Adding a penalty on large coefficients.
Train-test gap — The difference between training and test error; a large gap signals overfitting.
Train error 0.00 vs test error 0.30.
