Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 1: Data Preparation and Model Foundations

1.8 Normalizing Numerical Features

Print / Save as PDF

1.8 Normalizing Numerical Features

After this unit you will: be able to apply the two scaling formulas and not confuse them with each other.

Key points

Normalization (Min-Max) — brings values into the interval [0, 1]:

x_new = (x_old − x_min) / (x_max − x_min)

Standardization (z-score) — brings them to mean 0 and standard deviation 1:

x_new = (x_old − mean) / std

  • Example on the slide: the grade column (6, 1, 8, 0, 3, 10, 2) becomes (0.6, 0.1, 0.8, 0.0, 0.3, 1.0, 0.2) after Min-Max.
  • After z-score, values can be negative (−0.84, −1.14 ...) and are not bounded to [0, 1].
  • The two transformations are not interchangeable — whichever you use, you must be able to explain why.

Study notes

By the end of this lesson you can:

  • Explain why features on very different scales can mislead a machine learning algorithm.
  • Apply the Min-Max normalization formula by hand and with MinMaxScaler.
  • Apply the z-score standardization formula by hand and with StandardScaler.
  • Choose between normalization and standardization and justify the choice.

Key idea: Many algorithms are sensitive to the scale of features. Normalization squeezes values into [0, 1]; standardization gives them mean 0 and standard deviation 1. They are different transformations, so choose one deliberately and be able to explain why.

Why scaling is needed

Imagine one feature measured in the thousands, such as salary, and another in single digits, such as number of children. Without scaling, an algorithm may treat salary as far more important simply because its numbers are bigger, not because it predicts better. Scaling puts every feature on the same playing field so the algorithm can judge them by their real predictive value.

Normalization (Min-Max scaling)

Normalization rescales a numerical feature into a fixed range, usually 0 to 1:

x_new = (x_old - x_min) / (x_max - x_min)

Subtract the column's minimum, then divide by the range (maximum minus minimum). In the lecture, a grade column runs from 0 to 10:

GradeCalculationNormalized
6(6 - 0) / (10 - 0)0.6
10 (the maximum)(10 - 0) / (10 - 0)1.0
0 (the minimum)(0 - 0) / (10 - 0)0.0

The minimum always becomes 0 and the maximum always becomes 1.

Standardization (z-score)

Standardization does not force a fixed range. It shifts and rescales the data so it has mean 0 and standard deviation 1:

x_new = (x_old - mean) / std

The standard deviation measures how spread out the values are. Values above the mean become positive, values below the mean become negative, and the results are not limited to [0, 1].

Choosing between them

Normalization (Min-Max)Standardization (z-score)
OutputFixed range, usually [0, 1]Mean 0, std 1, unbounded
Often preferred whenData does not follow a bell curve, or the algorithm expects a bounded input rangeData roughly follows a bell curve (normal distribution)
OutliersBadly skewed: one extreme min or max squashes the other values togetherHandles outliers somewhat better
scikit-learnMinMaxScalerStandardScaler

In practice you will often try both and keep the one that gives better model performance for your problem.

Worked example

Scale the lecture's grade column with both scalers. scikit-learn expects a 2-D input (rows by columns), so the grades are written as a column.

import numpy as np
from sklearn.preprocessing import MinMaxScaler, StandardScaler

grade = np.array([[6], [1], [8], [0], [3], [10], [2]], dtype=float)

normalized = MinMaxScaler().fit_transform(grade).ravel()
standardized = StandardScaler().fit_transform(grade).ravel()

print(normalized)
print(standardized.round(2))
print(round(grade.mean(), 3), round(grade.std(), 3))

Expected output:

[0.6 0.1 0.8 0.  0.3 1.  0.2]
[ 0.49 -0.94  1.06 -1.23 -0.37  1.64 -0.65]
4.286 3.493

The normalized values match the lecture: each grade divided by 10. For standardization the mean is about 4.286 and the standard deviation about 3.493, so a grade of 6 becomes (6 - 4.286) / 3.493 = 0.49, while grades below the mean, such as 1 and 0, become negative.

Common mistakes

  • Mixing up the formulas. Min-Max uses the minimum and maximum; z-score uses the mean and standard deviation.
  • Expecting z-scores to lie in [0, 1]. Standardized values are often negative and can exceed 1.
  • Using Min-Max on data with an extreme outlier. One huge value becomes 1 and pushes all the normal values close to 0.
  • Calculating std by hand with pandas and getting a different answer. StandardScaler uses the population standard deviation (np.std), while Series.std() in pandas uses the sample version, so the numbers differ slightly.

Summary

  • Unscaled features let large-valued columns dominate for the wrong reason.
  • Normalization: (x - min) / (max - min), giving values in [0, 1].
  • Standardization: (x - mean) / std, giving mean 0 and std 1, with negative values possible.
  • Min-Max is sensitive to outliers; z-score handles them a little better.
  • Use MinMaxScaler or StandardScaler, and be able to explain your choice.

Flashcards · 1.8 Normalizing Numerical Features

8 words
WordMeaningExample
  • Feature scaling

    Transforming numerical features to comparable ranges so no feature dominates just because its numbers are larger.

    Salary in millions and children in single digits.

  • Normalization (Min-Max)

    Rescales values into a fixed range, usually [0, 1], using the column's minimum and maximum.

    x_new = (x - x_min) / (x_max - x_min)

  • Standardization (z-score)

    Rescales values so the feature has mean 0 and standard deviation 1.

    x_new = (x - mean) / std

  • Standard deviation

    A measure of how spread out the values of a feature are around their mean.

    np.std([6, 1, 8, 0, 3, 10, 2]) is about 3.49

  • MinMaxScaler

    scikit-learn transformer that applies Min-Max normalization to each column.

    MinMaxScaler().fit_transform(grade)

  • StandardScaler

    scikit-learn transformer that standardizes each column to mean 0 and standard deviation 1.

    StandardScaler().fit_transform(grade)

  • Bell curve

    The shape of a normal distribution; standardization is often preferred when data roughly follows it.

    Heights of adults in a large population.

  • Outlier sensitivity

    Min-Max scaling is badly skewed by one extreme value, which squashes the other values together.

    Incomes 10, 12, 13 and 110: the normal ones end up near 0.