Back to the lessonPreparing the printout…

Fundamental of Machine Learning · Module 3: Model Evaluation and Optimization

Capstone Practice: Choosing Evaluation Metrics for Three Scenarios

[markdown] content available only on the platform

[markdown] content available only on the platform

Flashcards · Capstone Practice: Choosing Evaluation Metrics for Three Scenarios

Cost of errors — The real-world consequence of a false positive or a false negative, which drives the choice of metric.

A missed disease case costs far more than a false alarm.

Recall-first scenario — A situation where missing a positive case is the most expensive error.

Cancer screening.

Precision-first scenario — A situation where a false alarm is the most expensive error.

A spam filter blocking a contract email.

F1-first scenario — A situation where both false alarms and misses are costly, so a balanced metric is needed.

Fraud detection.

Misleading accuracy — High accuracy on imbalanced data that hides a model's failure on the minority class.

96% accuracy, 0 returns flagged.

MAE for stakeholders — MAE states the average error in the target's own unit, which non-technical people understand.

Off by 0.3 billion VND on average.

R-squared for model value — R-squared shows how much better the model is than always predicting the mean.

R-squared 0.8 vs the mean baseline at 0.

zero_division — classification_report argument that sets the value shown when a metric divides by zero, avoiding warnings.

classification_report(y, y_pred, zero_division=0)

Fundamental of Machine Learning · Capstone Practice: Choosing Evaluation Metrics for Three Scenarios · printed on November 21, 2026