You are viewing a public preview
Sign in to save your progress, do assignments, join discussions and get feedback.
After this lesson you will be able to: compute MSE and RMSE both by hand and with scikit-learn, understand why RMSE has the same unit as the target variable, and see how much more sensitive MSE/RMSE are to outliers than MAE.
For n samples, let the error of the i-th sample be e = y_pred − y_true:
| Metric | Formula | Unit |
|---|---|---|
| MAE | mean of the absolute value abs(e) | same unit as y |
| MSE | mean of e² | the square of the unit of y |
| RMSE | square root of MSE | same unit as y |
| True price | Predicted price | e | abs(e) | e² |
|---|---|---|---|---|
| 2.0 | 2.5 | +0.5 | 0.5 | 0.25 |
| 3.0 | 2.5 | −0.5 | 0.5 | 0.25 |
| 4.0 | 4.0 | 0 | 0 | 0 |
| 5.0 | 4.0 | −1.0 | 1.0 | 1.0 |
| 6.0 | 7.0 | +1.0 | 1.0 | 1.0 |
The sum of abs(e) = 3, so MAE = 3 / 5 = 0.6. The sum of e² = 2.5, so MSE = 2.5 / 5 = 0.5 and RMSE = √0.5 ≈ 0.707. Check with NumPy:
import numpy as np
y_true = np.array([2.0, 3.0, 4.0, 5.0, 6.0]) # true prices (billion VND)
y_pred = np.array([2.5, 2.5, 4.0, 4.0, 7.0]) # prices predicted by the model
err = y_pred - y_true
mae = np.mean(np.abs(err))
mse = np.mean(err ** 2)
print(mae, mse, round(np.sqrt(mse), 4)) # 0.6 0.5 0.7071
from sklearn.metrics import (mean_absolute_error, mean_squared_error,
root_mean_squared_error, r2_score)
print(mean_absolute_error(y_true, y_pred)) # 0.6
print(mean_squared_error(y_true, y_pred)) # 0.5
print(root_mean_squared_error(y_true, y_pred)) # 0.7071067811865476
print(np.sqrt(mean_squared_error(y_true, y_pred))) # 0.7071067811865476
print(r2_score(y_true, y_pred)) # 0.75
root_mean_squared_error has existed since scikit-learn 1.4; before that, people wrote np.sqrt(mean_squared_error(...)). Do not use the squared=False parameter from older documentation: newer versions removed it and raise a TypeError. Always put y_true before y_pred.
Unit: RMSE = 0.707 billion VND, meaning the "typical" error is about 0.7 billion. MSE = 0.5 but its unit is billion², which is hard to interpret directly. To judge whether 0.707 is good or bad, compare it with the model that always predicts the mean: in that case the MSE is exactly the variance of y (np.var(y_true) = 2.0) and the RMSE = √2 ≈ 1.414. Our model halves it, which matches R² = 1 − 0.5 / 2.0 = 0.75.
y_bad = y_pred.copy()
y_bad[-1] = 11.0 # the last house is off by 5 billion
print(mean_absolute_error(y_true, y_bad)) # 1.4
print(mean_squared_error(y_true, y_bad)) # 5.3
print(root_mean_squared_error(y_true, y_bad)) # 2.3021728866442674
With just one large error: MAE rises from 0.6 to 1.4 (2.3 times as large), but RMSE rises from 0.707 to 2.30 (3.3 times as large), because the error of 5 is squared into 25. RMSE is always ≥ MAE; if RMSE is much larger than MAE, the model has a few very large errors.
root_mean_squared_error (scikit-learn ≥ 1.4) or np.sqrt(mean_squared_error(...)).Flashcards · 3.6 MSE and RMSE
9 wordsThe average of the squared prediction errors; its unit is the square of the target's unit.
The square root of MSE, which brings the error back to the same unit as the target.
Squaring makes big errors count disproportionately: an error 10 times larger contributes 100 times more.
RMSE is always at least MAE; a much larger RMSE signals a few very large errors.
One large error raises RMSE far more than MAE, so RMSE suits cases where big mistakes are costly.
scikit-learn function computing MSE; always pass y_true before y_pred.
scikit-learn function (version 1.4 and later) that computes RMSE directly.
An old way to get RMSE from mean_squared_error; newer scikit-learn versions removed it and raise a TypeError.
Always predicting the mean gives an MSE equal to the variance of y, a reference for judging MSE.