[markdown] content available only on the platform
Flashcards · 3.7 Lab: Cross-Validation and GridSearchCV
KFold — Splits data into K folds; each sample is validated once and trained on K-1 times. It does not shuffle by default.
KFold(n_splits=5).split(X10)
Fold sizes — When samples do not divide evenly, KFold makes the first folds one sample larger.
10 samples, 3 folds -> sizes 4, 3, 3.
cross_val_score() — Trains and scores a model on each fold, returning one score per fold.
cross_val_score(model, X, y, cv=5) -> 5 scores
Mean and standard deviation of CV scores — The mean estimates expected performance; the standard deviation shows how much results vary between splits.
0.973 +/- 0.025
Shuffling with sorted labels — Data sorted by label needs shuffled or stratified folds, otherwise a fold may contain a class never seen in training.
KFold(n_splits=3, shuffle=True, random_state=0)
StratifiedKFold — Fold splitter that keeps class proportions in each fold; used automatically when cv=5 is given for a classifier.
cross_val_score(clf, X, y, cv=5)
GridSearchCV — Tries every hyperparameter combination with cross-validation on the training set and keeps the best one.
GridSearchCV(DecisionTreeClassifier(), grid, cv=5)
Number of fits — GridSearchCV trains combinations x folds models, plus one refit of the best combination.
4 x 2 combinations x 5 folds = 40 fits, + 1 refit.
refit=True — Default GridSearchCV setting that retrains the best combination on the whole training set as best_estimator_.
search.best_estimator_.predict(X_test)
Test set used once — Choose hyperparameters on training data only; score the final model on the test set exactly once.
search.score(X_test, y_test) at the very end.
[file] /api/lesson-media/files/9b417aa2-5e9f-4ae3-a9db-2a3b7f03f372-1794890094129-268e09a859caacc4.zip