Back to the lessonPreparing the printout…

Fundamental of Machine Learning · Module 3: Model Evaluation and Optimization

Hyperparameter Tuning

[markdown] content available only on the platform

[markdown] content available only on the platform

Flashcards · Hyperparameter Tuning

Parameter — A value the model learns from the training data.

The coefficient w in Linear Regression.

Hyperparameter — A setting chosen by the user before training that controls how the model learns.

max_depth, C, n_estimators

Grid search — Trying every combination of listed hyperparameter values and keeping the best by cross-validation.

GridSearchCV(pipe, grid, cv=5)

Number of trainings — Grid search trains combinations x folds models (plus one refit).

4 x 2 combinations x 5 folds = 40.

Random search — Trying a fixed number of randomly drawn hyperparameter combinations; often more efficient than a grid.

RandomizedSearchCV(pipe, grid, n_iter=20, cv=5)

best_params_ — The hyperparameter combination with the highest mean cross-validation score.

{'model__max_depth': 5, 'model__n_estimators': 300}

Step prefix (double underscore) — In a pipeline grid, a hyperparameter is named step name + __ + parameter.

'model__max_depth'

Coarse-to-fine search — Search a wide range with few values first, then search densely around the best region.

[2, 5, 10, 20] then [4, 5, 6, 7].

Honest test score — A test score from a model whose hyperparameters were chosen without looking at the test set.

search.score(x_test, y_test) run once at the end.

Fundamental of Machine Learning · Hyperparameter Tuning · printed on November 21, 2026