Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 3: Model Evaluation and Optimization

3.6 MSE and RMSE

Print / Save as PDF

MSE and RMSE

After this lesson you will be able to: compute MSE and RMSE both by hand and with scikit-learn, understand why RMSE has the same unit as the target variable, and see how much more sensitive MSE/RMSE are to outliers than MAE.

1. Definitions

For n samples, let the error of the i-th sample be e = y_pred − y_true:

MetricFormulaUnit
MAEmean of the absolute value abs(e)same unit as y
MSEmean of e²the square of the unit of y
RMSEsquare root of MSEsame unit as y

2. Hand calculation with 5 houses (unit: billion VND)

True pricePredicted priceeabs(e)e²
2.02.5+0.50.50.25
3.02.5−0.50.50.25
4.04.0000
5.04.0−1.01.01.0
6.07.0+1.01.01.0

The sum of abs(e) = 3, so MAE = 3 / 5 = 0.6. The sum of e² = 2.5, so MSE = 2.5 / 5 = 0.5 and RMSE = √0.5 ≈ 0.707. Check with NumPy:

import numpy as np

y_true = np.array([2.0, 3.0, 4.0, 5.0, 6.0])   # true prices (billion VND)
y_pred = np.array([2.5, 2.5, 4.0, 4.0, 7.0])   # prices predicted by the model

err = y_pred - y_true
mae = np.mean(np.abs(err))
mse = np.mean(err ** 2)
print(mae, mse, round(np.sqrt(mse), 4))   # 0.6 0.5 0.7071

3. Using scikit-learn

from sklearn.metrics import (mean_absolute_error, mean_squared_error,
                             root_mean_squared_error, r2_score)

print(mean_absolute_error(y_true, y_pred))          # 0.6
print(mean_squared_error(y_true, y_pred))           # 0.5
print(root_mean_squared_error(y_true, y_pred))      # 0.7071067811865476
print(np.sqrt(mean_squared_error(y_true, y_pred)))  # 0.7071067811865476
print(r2_score(y_true, y_pred))                     # 0.75

root_mean_squared_error has existed since scikit-learn 1.4; before that, people wrote np.sqrt(mean_squared_error(...)). Do not use the squared=False parameter from older documentation: newer versions removed it and raise a TypeError. Always put y_true before y_pred.

Unit: RMSE = 0.707 billion VND, meaning the "typical" error is about 0.7 billion. MSE = 0.5 but its unit is billion², which is hard to interpret directly. To judge whether 0.707 is good or bad, compare it with the model that always predicts the mean: in that case the MSE is exactly the variance of y (np.var(y_true) = 2.0) and the RMSE = √2 ≈ 1.414. Our model halves it, which matches R² = 1 − 0.5 / 2.0 = 0.75.

4. Outliers: MSE/RMSE penalize them more heavily than MAE

y_bad = y_pred.copy()
y_bad[-1] = 11.0                                     # the last house is off by 5 billion
print(mean_absolute_error(y_true, y_bad))            # 1.4
print(mean_squared_error(y_true, y_bad))             # 5.3
print(root_mean_squared_error(y_true, y_bad))        # 2.3021728866442674

With just one large error: MAE rises from 0.6 to 1.4 (2.3 times as large), but RMSE rises from 0.707 to 2.30 (3.3 times as large), because the error of 5 is squared into 25. RMSE is always ≥ MAE; if RMSE is much larger than MAE, the model has a few very large errors.

Common Mistakes

  • Reading MSE as if it had the same unit as y (it actually has the squared unit).
  • Assuming a smaller MSE is always better without comparing against a baseline or the scale of y.
  • Using RMSE without thinking about outliers: a few noisy points are enough to make it shoot up, whereas MAE is more stable.

Key Points

  • MSE = mean of e²; RMSE = √MSE, the same unit as y.
  • Squaring penalizes large errors heavily: RMSE is more sensitive to outliers than MAE.
  • root_mean_squared_error (scikit-learn ≥ 1.4) or np.sqrt(mean_squared_error(...)).

Flashcards · 3.6 MSE and RMSE

9 words
WordMeaningExample
  • Mean squared error (MSE)

    The average of the squared prediction errors; its unit is the square of the target's unit.

    Errors 0.5, -0.5, 0, -1, 1 -> MSE = 2.5 / 5 = 0.5

  • Root mean squared error (RMSE)

    The square root of MSE, which brings the error back to the same unit as the target.

    sqrt(0.5) = 0.707 billion VND

  • Squaring penalises large errors

    Squaring makes big errors count disproportionately: an error 10 times larger contributes 100 times more.

    0.1^2 = 0.01 but 1^2 = 1.

  • RMSE vs MAE

    RMSE is always at least MAE; a much larger RMSE signals a few very large errors.

    MAE 1.4 vs RMSE 2.30 after one 5-billion error.

  • Outlier sensitivity of RMSE

    One large error raises RMSE far more than MAE, so RMSE suits cases where big mistakes are costly.

    One bad house: MAE x2.3, RMSE x3.3.

  • mean_squared_error()

    scikit-learn function computing MSE; always pass y_true before y_pred.

    mean_squared_error(y_true, y_pred) -> 0.5

  • root_mean_squared_error()

    scikit-learn function (version 1.4 and later) that computes RMSE directly.

    root_mean_squared_error(y_true, y_pred) -> 0.7071

  • squared=False (removed)

    An old way to get RMSE from mean_squared_error; newer scikit-learn versions removed it and raise a TypeError.

    Use np.sqrt(mean_squared_error(...)) instead.

  • MSE of the mean baseline

    Always predicting the mean gives an MSE equal to the variance of y, a reference for judging MSE.

    np.var(y_true) = 2.0, so R-squared = 1 - 0.5 / 2.0 = 0.75