Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 3: Model Evaluation and Optimization

3.3 Precision, Recall, and F1 Score

Print / Save as PDF

3.3 Precision, Recall, and F1 Score

After this unit you will be able to: compute Precision, Recall, and F1 from a confusion matrix and know what question each metric answers.

Key points

MetricFormula
Accuracy(TP + TN) / All
PrecisionTP / (TP + FP)
RecallTP / (TP + FN)
F1 score2 × P × R / (P + R)
  • Precision: of the samples the model flagged as positive, what fraction really are positive.
  • Recall: of the samples that are truly positive, what fraction the model found.
  • F1 combines the two metrics with the formula 2PR/(P+R), so if either one is low, F1 is pulled down sharply — unlike the arithmetic mean.

Study notes

By the end of this lesson you can:

  • Compute precision, recall and F1 from a confusion matrix
  • State the question each metric answers
  • Decide when precision matters most (costly false positives) and when recall matters most (costly false negatives)
  • Explain why F1 uses the harmonic mean instead of the arithmetic mean
  • Recite the summary table of accuracy, precision, recall and F1

Key idea: Precision asks "when the model says yes, how often is it right?", recall asks "of all real yeses, how many did it catch?", and F1 combines both into one number that stays low if either is low.

Precision

precision = TP / (TP + FP)

In plain English: out of everything the model predicted as positive, what fraction was actually positive? With the dog and cat example (TP = 3, FP = 1):

precision = 3 / (3 + 1) = 0.75

Precision answers: when my model says "dog", how often can I trust it? High precision means few false alarms. It matters most when a false positive is costly. For a spam filter, low precision means important emails wrongly flagged as spam, which is annoying and possibly costly for the user.

Recall

recall = TP / (TP + FN)

In plain English: out of everything that was actually positive, what fraction did the model catch? With TP = 3 and FN = 2:

recall = 3 / (3 + 2) = 0.6

Recall answers: out of all the actual dogs, how many did my model find? High recall means the model rarely misses a positive case. It matters most when a false negative is costly. For disease detection we want very high recall, because missing a sick patient can have serious, even life-threatening, consequences. We would rather accept a few false alarms (lower precision) than miss real cases (lower recall).

A model with high precision but low recall is trustworthy when it says "positive", but misses many real positives. For a cancer-diagnosis model this would mean many patients with cancer are not detected.

F1 score

Precision and recall often trade off: improving one can hurt the other. To get one balanced number, use the F1 score:

F1 = 2 * precision * recall / (precision + recall)

F1 is the harmonic mean of precision and recall. This kind of average is very sensitive to low values: if either precision or recall is very low, F1 is also very low, even if the other is very high. That makes it a fairer summary than a simple average.

For the example: F1 = 2 * 0.75 * 0.6 / (0.75 + 0.6) = 0.9 / 1.35, about 0.67 (two thirds).

F1 is especially useful with imbalanced classes, where accuracy can be misleading: it forces you to balance catching real positives against avoiding false alarms.

The summary table

MetricFormulaQuestion it answers
Accuracy(TP + TN) / allOverall, how often am I right?
PrecisionTP / (TP + FP)When I said yes, how often was I right?
RecallTP / (TP + FN)Of everyone who was actually a yes, how many did I catch?
F1 score2 * P * R / (P + R)One balanced number combining P and R

The lecture recommends pausing to make sure you can recite these from memory.

Worked example

Check the dog and cat numbers with scikit-learn, then compare F1 with the arithmetic mean for an extreme model.

from sklearn.metrics import precision_score, recall_score, f1_score

actual    = ["dog", "dog", "cat", "dog", "cat", "dog", "dog"]
predicted = ["dog", "cat", "dog", "dog", "cat", "cat", "dog"]

p = precision_score(actual, predicted, pos_label="dog")
r = recall_score(actual, predicted, pos_label="dog")
f1 = f1_score(actual, predicted, pos_label="dog")
print(round(p, 2), round(r, 2), round(f1, 2))

# An extreme model: precision 0.95, recall 0.05
P, R = 0.95, 0.05
print("arithmetic mean", (P + R) / 2)
print("F1", round(2 * P * R / (P + R), 3))

Expected output:

0.75 0.6 0.67
arithmetic mean 0.5
F1 0.095

Step by step: the first line matches the lecture. For the extreme model, the arithmetic mean of 0.5 sounds acceptable, but F1 of about 0.1 reveals that the model misses almost every positive case. pos_label="dog" tells scikit-learn which class is positive when labels are strings.

Common mistakes

  • Swapping the denominators. Precision divides by everything predicted positive (TP + FP); recall divides by everything actually positive (TP + FN).
  • Using the arithmetic mean instead of F1. The simple average hides a very low precision or recall.
  • Optimising precision for a screening task. When misses are dangerous (disease), recall should come first.
  • Forgetting pos_label with string labels. scikit-learn needs to know which class counts as positive.

Summary

  • Precision = TP / (TP + FP): how trustworthy a positive prediction is; prioritise when false alarms are costly.
  • Recall = TP / (TP + FN): how many real positives are found; prioritise when misses are costly.
  • For the seven photos: precision 0.75, recall 0.6, F1 about 0.67.
  • F1 is the harmonic mean of precision and recall and drops sharply if either is low.
  • Accuracy, precision, recall and F1 are the core toolkit for classification models.

Flashcards · 3.3 Precision, Recall, and F1 Score

8 words
WordMeaningExample
  • Precision

    TP / (TP + FP): of all samples predicted positive, the fraction that are truly positive.

    3 / (3 + 1) = 0.75

  • Recall

    TP / (TP + FN): of all truly positive samples, the fraction the model found.

    3 / (3 + 2) = 0.6

  • F1 score

    The harmonic mean of precision and recall: 2PR / (P + R).

    2 * 0.75 * 0.6 / 1.35 = 0.67

  • Harmonic mean

    An average that is pulled strongly towards the smaller value, so one low input makes the result low.

    P = 1.0, R = 0 -> F1 = 0, not 0.5.

  • Precision-recall trade-off

    Changes that raise precision often lower recall, and vice versa.

    Flagging fewer emails as spam: fewer false alarms, more misses.

  • When to prioritise precision

    When false positives are costly, such as blocking legitimate emails.

    Spam filtering.

  • When to prioritise recall

    When false negatives are costly, such as missing a sick patient.

    Disease screening.

  • pos_label

    Argument telling scikit-learn metric functions which label is the positive class.

    precision_score(y, y_pred, pos_label='dog')