3.1 Accuracy and Imbalanced Data
[video] /api/lesson-media/videos/9b417aa2-5e9f-4ae3-a9db-2a3b7f03f372-1794614933263-5f9acdfcc5255af9.mp4
[markdown] content available only on the platform
[markdown] content available only on the platform
Flashcards · 3.1 Accuracy and Imbalanced Data
Binary classification — A problem with exactly two possible categories for each sample.
Is this photo a dog or a cat?
Accuracy — The proportion of predictions that are correct out of all predictions.
4 correct out of 7 -> accuracy about 0.57.
Imbalanced data — A dataset in which one class vastly outnumbers the other(s).
99% normal transactions, 1% fraud.
Majority class — The class with by far the most samples in an imbalanced dataset.
Healthy patients in disease screening.
Minority class — The rare class in an imbalanced dataset, often the one the model is built to find.
Fraudulent transactions (1%).
Accuracy paradox — A useless model can have very high accuracy on imbalanced data by always predicting the majority class.
Always 'no disease' -> 99% accuracy, 0 patients found.
accuracy_score() — The scikit-learn function that computes accuracy from true and predicted labels.
accuracy_score(y_true, y_pred)
Majority-class baseline — The accuracy obtained by always predicting the majority class; any real model must beat it.
990 of 1,000 normal -> baseline accuracy 0.99.
