Cross-Validation
[markdown] content available only on the platform
[markdown] content available only on the platform
Flashcards · Cross-Validation
Cross-validation — Evaluating a model several times on different train/validation splits and combining the scores.
cross_val_score(pipe, x_train, y_train, cv=5)
K-fold — Split data into K folds; each round validates on one fold and trains on the other K - 1.
K = 5: 2,000 rows -> train on 1,600, validate on 400.
Fold — One of the K equal parts the data is divided into for cross-validation.
With cv=5, each fold holds 20% of the rows.
CV standard deviation — The spread of fold scores; a large spread means the estimate depends heavily on the split.
0.80 +/- 0.15 is not yet trustworthy.
Stratified K-fold — K-fold that keeps the class proportions in every fold; automatic for classifiers in cross_val_score.
StratifiedKFold(n_splits=5)
Leave-one-out (LOO) — Cross-validation with K equal to the number of samples; least biased but very slow.
Only for small datasets.
Data leakage in CV — Fitting preprocessing on all data before CV lets validation information into training, inflating scores.
StandardScaler fitted before cross_val_score.
Pipeline — A scikit-learn object chaining preprocessing and a model so each step is fitted on training data only.
Pipeline([('scaler', StandardScaler()), ('model', SVC())])
