Data Partitioning
[markdown] content available only on the platform
[markdown] content available only on the platform
Flashcards · Data Partitioning
Overfitting — When a model memorises the training data, scoring very well on it but poorly on new data.
100% training accuracy, 60% test accuracy.
train_test_split() — scikit-learn function that randomly divides features and target into training and test parts.
train_test_split(x, y, test_size=0.2, random_state=1009)
test_size — Argument of train_test_split giving the share (or number) of rows placed in the test set.
test_size=0.3 on 500 rows gives 150 test rows.
random_state — Fixes the random split so rerunning the code produces exactly the same sets.
random_state=1009
stratify — Argument that keeps the class proportions the same in the training and test sets.
train_test_split(x, y, stratify=y)
Imbalanced data — A classification dataset where one class is much rarer than the others.
1% fraud, 99% normal transactions.
Test set reuse — Checking the test score repeatedly while tuning, which leaks information and inflates the final estimate.
Ten rounds of tweak, test, tweak.
Validation set — Data held out from training and used to choose models and tune hyperparameters.
10-20% of the data, or cross-validation folds.
