Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 3: Model Evaluation and Optimization

3.P6 Reading the Evaluation Results

Print / Save as PDF

3.P6 Reading the Evaluation Results

After this unit you will: read the precision, recall, f1-score, and support of each class and cross-check them against the confusion matrix.

Key points

Classprecisionrecallf1-scoresupport
0 (no disease)0.790.880.83100
1 (disease)0.710.560.6254
accuracy0.77154
  • confusion_matrix(y_test, y_pred) gives [[88, 12], [24, 30]], in the order [[TN, FP], [FN, TP]].
  • Class 1: a precision of 0.71 means that when the model says "disease", it is correct 71% of the time; a recall of 0.56 = 30 / (30 + 24) means the model finds only 56% of the true disease cases.
  • An accuracy of 0.77 hides the weakness on class 1 — for a disease-screening problem, missing nearly half of the disease cases (24 cases) is serious.
  • Ask yourself: is a recall of 0.56 acceptable?

Study notes

By the end of this lesson you can:

  • Read every column of a classification_report: precision, recall, f1-score and support
  • Explain the difference between macro average and weighted average and compute both
  • Read a scikit-learn confusion matrix (rows = truth, columns = prediction)
  • Cross-check the report against the confusion matrix
  • Judge whether the model is good enough for disease screening, beyond its accuracy

Key idea: An overall accuracy of 0.77 hides a weak spot: the model finds only 56% of the patients who really have diabetes; reading per-class metrics and the confusion matrix reveals it.

The classification report

Running the report for the diabetes model gives:

Classprecisionrecallf1-scoresupport
0 (no diabetes)0.790.880.83100
1 (diabetes)0.710.560.6254
accuracy0.77154
macro avg0.750.720.73154
weighted avg0.760.770.76154
  • Precision, recall, f1-score are the metrics from the theory part.
  • Support is simply the number of true samples of each class in the test set: 100 patients without diabetes and 54 with diabetes. It is not the number of predictions.
  • Each class row treats that class as the positive class: row 0 computes the metrics with class 0 as positive, row 1 with class 1 as positive.
  • Accuracy is a single overall value for the whole model.

Macro average versus weighted average

Macro average is a non-weighted mean over the classes. Each class counts equally, whatever its size:

  • precision: (0.79 + 0.71) / 2 = 0.75
  • recall: (0.88 + 0.56) / 2 = 0.72
  • f1-score: (0.83 + 0.62) / 2 = 0.73 (rounded)

Weighted average weights each class by its support, so the larger class contributes more:

  • precision: (0.79 x 100 + 0.71 x 54) / 154 = about 0.76

Use macro average when the minority class matters as much as the majority class; the weighted average tends to look closer to the majority class's performance.

The confusion matrix

from sklearn.metrics import confusion_matrix
print(confusion_matrix(y_test, y_pred))
# [[88 12]
#  [24 30]]

In scikit-learn the rows are the ground truth and the columns are the predictions, in the order 0 then 1, so the layout is [[TN, FP], [FN, TP]]:

  • 88 patients without diabetes correctly predicted healthy (TN);
  • 12 patients without diabetes wrongly predicted to have diabetes (FP);
  • 24 patients with diabetes wrongly predicted healthy (FN);
  • 30 patients with diabetes correctly detected (TP).

The four cells add up to 154, the size of the test set.

What the numbers mean for diabetes screening

For class 1, precision 0.71 means that when the model says "diabetes", it is right 71% of the time. Recall 0.56 = 30 / (30 + 24) means the model finds only 56% of the real diabetes cases. Missing 24 of 54 patients is serious in a screening context. So the honest question is not "is 0.77 accuracy good?" but "is a recall of 0.56 acceptable?" Usually, it is not.

Worked example

Rebuild the report from the confusion matrix to make sure you understand every number.

import numpy as np

cm = np.array([[88, 12], [24, 30]])
tn, fp, fn, tp = cm.ravel()

prec0, rec0 = tn / (tn + fn), tn / (tn + fp)
prec1, rec1 = tp / (tp + fp), tp / (tp + fn)
support = cm.sum(axis=1)

print(round(prec0, 2), round(rec0, 2), round(prec1, 2), round(rec1, 2))
print(support, cm.sum())
print("accuracy", round((tn + tp) / cm.sum(), 2))
print("macro precision", round((prec0 + prec1) / 2, 2))
print("weighted precision", round((prec0 * support[0] + prec1 * support[1]) / support.sum(), 2))

Expected output:

0.79 0.88 0.71 0.56
[100  54] 154
accuracy 0.77
macro precision 0.75
weighted precision 0.76

Step by step: for class 0 the "positive" predictions are the first column (88 + 24 = 112), so precision = 88 / 112. Support comes from the row sums. Every number matches the report.

Common mistakes

  • Reading support as the number of predictions. It is the number of true samples of each class.
  • Trusting accuracy alone. 0.77 overall hides a recall of only 0.56 for the class that matters.
  • Mixing up rows and columns. In scikit-learn rows are true labels and columns are predictions.
  • Confusing macro and weighted averages. Macro treats classes equally; weighted favours the larger class.

Summary

  • The report gives precision, recall, f1-score per class, plus support (true counts), accuracy and two averages.
  • Macro average = plain mean over classes; weighted average = mean weighted by support.
  • The confusion matrix [[88, 12], [24, 30]] is [[TN, FP], [FN, TP]]: 12 false alarms, 24 missed cases.
  • Class 1 recall = 30 / 54 = 0.56: almost half of the diabetes cases are missed.
  • For screening, judge the model by recall on class 1, not by overall accuracy.

Flashcards · 3.P6 Reading the Evaluation Results

8 words
WordMeaningExample
  • Support

    The number of true samples of each class in the test set.

    Class 0: 100, class 1: 54.

  • Macro average

    The plain mean of a metric over classes, giving every class equal weight.

    (0.79 + 0.71) / 2 = 0.75

  • Weighted average

    The mean of a metric over classes weighted by each class's support.

    (0.79*100 + 0.71*54) / 154 = 0.76

  • Per-class row

    Each class row of the report treats that class as the positive class.

    Row 1: precision 0.71, recall 0.56.

  • scikit-learn confusion matrix layout

    Rows are true labels and columns are predictions; for 0/1 labels it is [[TN, FP], [FN, TP]].

    [[88, 12], [24, 30]]

  • False negative in screening

    A patient with the disease predicted as healthy; the most serious error in screening.

    24 diabetes patients missed.

  • Hidden weakness of accuracy

    A decent overall accuracy can conceal poor recall on the minority class.

    Accuracy 0.77 but class 1 recall 0.56.

  • cm.ravel()

    Flattens a 2x2 confusion matrix so its cells can be unpacked as tn, fp, fn, tp.

    tn, fp, fn, tp = cm.ravel()