Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 1: Data Preparation and Model Foundations

1.9 Encoding Categorical Features

Print / Save as PDF

1.9 Encoding Categorical Features

After this unit you will: be able to choose the right way to encode a categorical feature as numbers.

Key points

TypeEncoding methodExample on the slide
OrdinalOrdinal encoding — assign numbers by orderXS → 0, S → 1, M → 2, L → 3, XL → 4
NominalOne-hot encoding — one 0/1 column per categoryblack, blue, white, green → 4 columns
  • After one-hot encoding, each row has exactly one column equal to 1.
  • Do not use ordinal encoding for colors: 0, 1, 2, 3 would create an order that does not really exist.

Study notes

By the end of this lesson you can:

  • Encode an ordinal feature with ordinal encoding while keeping its natural order.
  • Encode a nominal feature with one-hot encoding.
  • Explain why ordinal encoding must not be used for nominal features such as colours.
  • Use OrdinalEncoder and OneHotEncoder in scikit-learn and read their output.

Key idea: Models need numbers, but the way you convert categories matters. Use ordinal encoding when the categories have a real order, and one-hot encoding when they do not, so you never invent a ranking that does not exist.

Ordinal encoding for ordered categories

Ordinal features, such as clothing size, have a natural ranking even though they are written as words. Ordinal encoding keeps that order by giving each category an increasing number according to its rank. In the lecture:

SizeCode
XS0
S1
M2
L3
XL4

So a size column reading S, XS, L, M, XL, XS, M becomes 1, 0, 3, 2, 4, 0, 2. The numbers mean something real: a bigger number is a bigger size. The model now works with numbers but still respects that L is bigger than S. In scikit-learn this is OrdinalEncoder.

One-hot encoding for unordered categories

A colour column with values black, blue, white and green has no natural order. One-hot encoding creates one new binary column per category. Each row gets a 1 in the column of its own colour and 0 everywhere else:

colourblackbluewhitegreen
black1000
blue0100
white0010
green0001

Every row has exactly one 1. A column with k distinct categories becomes k new columns. In scikit-learn this is OneHotEncoder, one of the most common preprocessing tools in real projects.

Why not ordinal encoding for colours?

Suppose you code black = 0, blue = 1, white = 2, green = 3. The model may then "learn" that green is three times blue, or that white lies between blue and green. These relationships are false; they come only from the arbitrary numbers we chose. One-hot encoding avoids the problem: every colour is equally different from every other colour and no ranking is implied.

Feature typeEncodingReason
Ordinal (S, M, L)Ordinal encodingThe order is real, so the numbers carry true information
Nominal (colours, cities)One-hot encodingNo order exists, so we must not invent one

Worked example

Encode size with OrdinalEncoder and colour with OneHotEncoder.

import pandas as pd
from sklearn.preprocessing import OrdinalEncoder, OneHotEncoder

shirts = pd.DataFrame({
    "size":  ["S", "XS", "L", "M", "XL", "XS", "M"],
    "color": ["black", "blue", "white", "green", "black", "blue", "green"],
})

# give the order explicitly, otherwise categories are sorted alphabetically
size_encoder = OrdinalEncoder(categories=[["XS", "S", "M", "L", "XL"]])
shirts["size_code"] = size_encoder.fit_transform(shirts[["size"]]).ravel().astype(int)
print(shirts["size_code"].tolist())

color_encoder = OneHotEncoder()
color_matrix = color_encoder.fit_transform(shirts[["color"]]).toarray().astype(int)
print(color_encoder.get_feature_names_out())
print(color_matrix[:4])

Expected output:

[1, 0, 3, 2, 4, 0, 2]
['color_black' 'color_blue' 'color_green' 'color_white']
[[1 0 0 0]
 [0 1 0 0]
 [0 0 0 1]
 [0 0 1 0]]

The size codes match the lecture. OneHotEncoder orders the new columns alphabetically (black, blue, green, white), so white is the last column here, unlike the order on the slide. Each of the first four rows (black, blue, white, green) has a single 1.

Common mistakes

  • Using ordinal encoding on nominal data. Numbers for colours or cities create a fake ranking.
  • Letting OrdinalEncoder choose the order. Without categories=... it sorts alphabetically (L, M, S, XL, XS), which breaks the real size order.
  • Forgetting how many columns one-hot adds. A column with many categories becomes many columns; plan for that.
  • Expecting one-hot columns in the slide's order. scikit-learn sorts categories; always check get_feature_names_out().

Summary

  • Ordinal encoding maps ordered categories to increasing numbers: XS 0, S 1, M 2, L 3, XL 4.
  • One-hot encoding turns a nominal column into one 0/1 column per category.
  • After one-hot encoding each row has exactly one 1.
  • Never give nominal categories ranked numbers.
  • Use OrdinalEncoder (with an explicit order) and OneHotEncoder in scikit-learn.

Flashcards · 1.9 Encoding Categorical Features

8 words
WordMeaningExample
  • Encoding

    Converting categorical values into numbers that a machine learning model can use.

    Turning "M" into 2.

  • Ordinal encoding

    Assigns increasing integers to ordered categories so the numbers keep the real order.

    XS 0, S 1, M 2, L 3, XL 4

  • One-hot encoding

    Creates one binary column per category, with 1 in the matching column and 0 elsewhere.

    blue -> [0, 1, 0, 0]

  • OrdinalEncoder

    scikit-learn transformer for ordinal encoding; pass categories to set the correct order.

    OrdinalEncoder(categories=[["XS", "S", "M", "L", "XL"]])

  • OneHotEncoder

    scikit-learn transformer that turns each category of a nominal column into its own 0/1 column.

    OneHotEncoder().fit_transform(df[["color"]])

  • Fake ordering

    A false ranking created when nominal categories are given arbitrary ordered numbers.

    black 0, green 3 suggests green is "bigger".

  • get_feature_names_out()

    Returns the names of the columns an encoder produces, so you can check their order.

    ['color_black' 'color_blue' 'color_green' 'color_white']

  • Binary column

    A column containing only 0 and 1, as produced for each category by one-hot encoding.

    color_white: 0, 0, 1, 0