You are viewing a public preview
Sign in to save your progress, do assignments, join discussions and get feedback.
After this unit you will: read the precision, recall, f1-score, and support of each class and cross-check them against the confusion matrix.
| Class | precision | recall | f1-score | support |
|---|---|---|---|---|
| 0 (no disease) | 0.79 | 0.88 | 0.83 | 100 |
| 1 (disease) | 0.71 | 0.56 | 0.62 | 54 |
| accuracy | 0.77 | 154 |
confusion_matrix(y_test, y_pred) gives [[88, 12], [24, 30]], in the order [[TN, FP], [FN, TP]].By the end of this lesson you can:
classification_report: precision, recall, f1-score and supportKey idea: An overall accuracy of 0.77 hides a weak spot: the model finds only 56% of the patients who really have diabetes; reading per-class metrics and the confusion matrix reveals it.
Running the report for the diabetes model gives:
| Class | precision | recall | f1-score | support |
|---|---|---|---|---|
| 0 (no diabetes) | 0.79 | 0.88 | 0.83 | 100 |
| 1 (diabetes) | 0.71 | 0.56 | 0.62 | 54 |
| accuracy | 0.77 | 154 | ||
| macro avg | 0.75 | 0.72 | 0.73 | 154 |
| weighted avg | 0.76 | 0.77 | 0.76 | 154 |
Macro average is a non-weighted mean over the classes. Each class counts equally, whatever its size:
Weighted average weights each class by its support, so the larger class contributes more:
Use macro average when the minority class matters as much as the majority class; the weighted average tends to look closer to the majority class's performance.
from sklearn.metrics import confusion_matrix
print(confusion_matrix(y_test, y_pred))
# [[88 12]
# [24 30]]
In scikit-learn the rows are the ground truth and the columns are the predictions, in the order 0 then 1, so the layout is [[TN, FP], [FN, TP]]:
The four cells add up to 154, the size of the test set.
For class 1, precision 0.71 means that when the model says "diabetes", it is right 71% of the time. Recall 0.56 = 30 / (30 + 24) means the model finds only 56% of the real diabetes cases. Missing 24 of 54 patients is serious in a screening context. So the honest question is not "is 0.77 accuracy good?" but "is a recall of 0.56 acceptable?" Usually, it is not.
Rebuild the report from the confusion matrix to make sure you understand every number.
import numpy as np
cm = np.array([[88, 12], [24, 30]])
tn, fp, fn, tp = cm.ravel()
prec0, rec0 = tn / (tn + fn), tn / (tn + fp)
prec1, rec1 = tp / (tp + fp), tp / (tp + fn)
support = cm.sum(axis=1)
print(round(prec0, 2), round(rec0, 2), round(prec1, 2), round(rec1, 2))
print(support, cm.sum())
print("accuracy", round((tn + tp) / cm.sum(), 2))
print("macro precision", round((prec0 + prec1) / 2, 2))
print("weighted precision", round((prec0 * support[0] + prec1 * support[1]) / support.sum(), 2))
Expected output:
0.79 0.88 0.71 0.56
[100 54] 154
accuracy 0.77
macro precision 0.75
weighted precision 0.76
Step by step: for class 0 the "positive" predictions are the first column (88 + 24 = 112), so precision = 88 / 112. Support comes from the row sums. Every number matches the report.
Flashcards · 3.P6 Reading the Evaluation Results
8 wordsThe number of true samples of each class in the test set.
The plain mean of a metric over classes, giving every class equal weight.
The mean of a metric over classes weighted by each class's support.
Each class row of the report treats that class as the positive class.
Rows are true labels and columns are predictions; for 0/1 labels it is [[TN, FP], [FN, TP]].
A patient with the disease predicted as healthy; the most serious error in screening.
A decent overall accuracy can conceal poor recall on the minority class.
Flattens a 2x2 confusion matrix so its cells can be unpacked as tn, fp, fn, tp.