Back to the lessonPreparing the printout…

Fundamental of Machine Learning · Module 3: Model Evaluation and Optimization

Cross-Validation

[markdown] content available only on the platform

[markdown] content available only on the platform

Flashcards · Cross-Validation

Cross-validation — Evaluating a model several times on different train/validation splits and combining the scores.

cross_val_score(pipe, x_train, y_train, cv=5)

K-fold — Split data into K folds; each round validates on one fold and trains on the other K - 1.

K = 5: 2,000 rows -> train on 1,600, validate on 400.

Fold — One of the K equal parts the data is divided into for cross-validation.

With cv=5, each fold holds 20% of the rows.

CV standard deviation — The spread of fold scores; a large spread means the estimate depends heavily on the split.

0.80 +/- 0.15 is not yet trustworthy.

Stratified K-fold — K-fold that keeps the class proportions in every fold; automatic for classifiers in cross_val_score.

StratifiedKFold(n_splits=5)

Leave-one-out (LOO) — Cross-validation with K equal to the number of samples; least biased but very slow.

Only for small datasets.

Data leakage in CV — Fitting preprocessing on all data before CV lets validation information into training, inflating scores.

StandardScaler fitted before cross_val_score.

Pipeline — A scikit-learn object chaining preprocessing and a model so each step is fitted on training data only.

Pipeline([('scaler', StandardScaler()), ('model', SVC())])

Fundamental of Machine Learning · Cross-Validation · printed on November 20, 2026