You are viewing a public preview
Sign in to save your progress, do assignments, join discussions and get feedback.
After this unit you will be able to: compute Precision, Recall, and F1 from a confusion matrix and know what question each metric answers.
| Metric | Formula |
|---|---|
| Accuracy | (TP + TN) / All |
| Precision | TP / (TP + FP) |
| Recall | TP / (TP + FN) |
| F1 score | 2 × P × R / (P + R) |
By the end of this lesson you can:
Key idea: Precision asks "when the model says yes, how often is it right?", recall asks "of all real yeses, how many did it catch?", and F1 combines both into one number that stays low if either is low.
precision = TP / (TP + FP)
In plain English: out of everything the model predicted as positive, what fraction was actually positive? With the dog and cat example (TP = 3, FP = 1):
precision = 3 / (3 + 1) = 0.75
Precision answers: when my model says "dog", how often can I trust it? High precision means few false alarms. It matters most when a false positive is costly. For a spam filter, low precision means important emails wrongly flagged as spam, which is annoying and possibly costly for the user.
recall = TP / (TP + FN)
In plain English: out of everything that was actually positive, what fraction did the model catch? With TP = 3 and FN = 2:
recall = 3 / (3 + 2) = 0.6
Recall answers: out of all the actual dogs, how many did my model find? High recall means the model rarely misses a positive case. It matters most when a false negative is costly. For disease detection we want very high recall, because missing a sick patient can have serious, even life-threatening, consequences. We would rather accept a few false alarms (lower precision) than miss real cases (lower recall).
A model with high precision but low recall is trustworthy when it says "positive", but misses many real positives. For a cancer-diagnosis model this would mean many patients with cancer are not detected.
Precision and recall often trade off: improving one can hurt the other. To get one balanced number, use the F1 score:
F1 = 2 * precision * recall / (precision + recall)
F1 is the harmonic mean of precision and recall. This kind of average is very sensitive to low values: if either precision or recall is very low, F1 is also very low, even if the other is very high. That makes it a fairer summary than a simple average.
For the example: F1 = 2 * 0.75 * 0.6 / (0.75 + 0.6) = 0.9 / 1.35, about 0.67 (two thirds).
F1 is especially useful with imbalanced classes, where accuracy can be misleading: it forces you to balance catching real positives against avoiding false alarms.
| Metric | Formula | Question it answers |
|---|---|---|
| Accuracy | (TP + TN) / all | Overall, how often am I right? |
| Precision | TP / (TP + FP) | When I said yes, how often was I right? |
| Recall | TP / (TP + FN) | Of everyone who was actually a yes, how many did I catch? |
| F1 score | 2 * P * R / (P + R) | One balanced number combining P and R |
The lecture recommends pausing to make sure you can recite these from memory.
Check the dog and cat numbers with scikit-learn, then compare F1 with the arithmetic mean for an extreme model.
from sklearn.metrics import precision_score, recall_score, f1_score
actual = ["dog", "dog", "cat", "dog", "cat", "dog", "dog"]
predicted = ["dog", "cat", "dog", "dog", "cat", "cat", "dog"]
p = precision_score(actual, predicted, pos_label="dog")
r = recall_score(actual, predicted, pos_label="dog")
f1 = f1_score(actual, predicted, pos_label="dog")
print(round(p, 2), round(r, 2), round(f1, 2))
# An extreme model: precision 0.95, recall 0.05
P, R = 0.95, 0.05
print("arithmetic mean", (P + R) / 2)
print("F1", round(2 * P * R / (P + R), 3))
Expected output:
0.75 0.6 0.67
arithmetic mean 0.5
F1 0.095
Step by step: the first line matches the lecture. For the extreme model, the arithmetic mean of 0.5 sounds acceptable, but F1 of about 0.1 reveals that the model misses almost every positive case. pos_label="dog" tells scikit-learn which class is positive when labels are strings.
Flashcards · 3.3 Precision, Recall, and F1 Score
8 wordsTP / (TP + FP): of all samples predicted positive, the fraction that are truly positive.
TP / (TP + FN): of all truly positive samples, the fraction the model found.
The harmonic mean of precision and recall: 2PR / (P + R).
An average that is pulled strongly towards the smaller value, so one low input makes the result low.
Changes that raise precision often lower recall, and vice versa.
When false positives are costly, such as blocking legitimate emails.
When false negatives are costly, such as missing a sick patient.
Argument telling scikit-learn metric functions which label is the positive class.