Đại học Kinh tế Quốc dân – National Economics UniversityMICRO4ASIA
🏆 Arena
Sign inSign up
MICRO4ASIAThe six micro-credentials

Courses are delivered on the Limio learning platform. Limio

← Fundamental of Machine Learning

You are viewing a public preview

Sign in to save your progress, do assignments, join discussions and get feedback.

Sign in
Module 1: Data Preparation and Model Foundations

Machine Learning Pipelines in Scikit-learn

Print / Save as PDF

Machine Learning Pipelines in Scikit-learn

The problem a Pipeline solves

When working manually, you must remember the exact order and the exact rules: fit the scaler on train, transform test, and only then train the model. Forgetting just once leaks data without any error message — the model still runs, the results still look great, they are just wrong.

Pipeline combines all the steps into a single object. Call fit once, and the library automatically applies the correct rule to each step.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier

pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("model",  RandomForestClassifier(random_state=100)),
])

pipe.fit(x_train, y_train)      # scaler.fit_transform + model.fit
y_pred = pipe.predict(x_test)   # scaler.transform + model.predict

Why this is the safer approach

When used together with cross-validation, the Pipeline refits the scaler separately for each fold. When working manually, people often standardize once on the entire training set and only then split into folds — and so every validation fold has already been leaked into.

from sklearn.model_selection import cross_val_score
scores = cross_val_score(pipe, x_train, y_train, cv=5)  # safe

Handling numeric and categorical columns differently

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder

pre = ColumnTransformer([
    ("num", StandardScaler(), ["age", "income"]),
    ("cat", OneHotEncoder(handle_unknown="ignore"), ["city"]),
])
pipe = Pipeline([("pre", pre), ("model", RandomForestClassifier())])

The most common mistake

Standardizing the data before passing it into cross-validation. The results will look better than reality and you will not notice, because nothing reports an error. Putting the scaler inside the Pipeline is the sure way to avoid it.

Study notes

By the end of this lesson you can:

  • Explain what problem a scikit-learn Pipeline solves.
  • Build a Pipeline that chains a scaler and a model, and call fit and predict on it.
  • Use a Pipeline with cross_val_score without leaking validation data.
  • Use ColumnTransformer to treat numerical and categorical columns differently.

Key idea: A Pipeline packs preprocessing and the model into one object, so every step automatically follows the rule "fit on training data, only transform new data". This prevents silent data leakage, especially inside cross-validation.

The problem a Pipeline solves

Working by hand, you must remember the exact order and the exact rules: fit the scaler on train, transform test, then train the model. Forget once and data leaks without any error message. The model still runs and the results still look great; they are simply wrong.

A Pipeline combines the steps into a single object. You call fit once and the library applies the correct rule at each step:

You callWhat the Pipeline does
pipe.fit(x_train, y_train)scaler.fit_transform(x_train), then model.fit(...) on the result
pipe.predict(x_test)scaler.transform(x_test), then model.predict(...)

A typical definition lists named steps in order: Pipeline([("scaler", StandardScaler()), ("model", RandomForestClassifier(random_state=100))]). All steps except the last must be transformers; the last step is the model.

Why it is safer with cross-validation

Cross-validation splits the training set into several folds and, in turn, uses each fold for validation. When a Pipeline is passed to cross_val_score(pipe, x_train, y_train, cv=5), the scaler is refit separately inside each fold, using only that fold's training part. Working by hand, people often standardise the whole training set once and only then split into folds. Every validation fold has then already influenced the scaling, and the scores look better than they really are.

Different treatment for numeric and categorical columns

Real tables mix column types. ColumnTransformer applies a different transformer to each group of columns and joins the results side by side:

  • ("num", StandardScaler(), ["age", "income"]) scales the numerical columns;
  • ("cat", OneHotEncoder(handle_unknown="ignore"), ["city"]) one-hot encodes the city column.

handle_unknown="ignore" matters in production: if a new customer comes from a city never seen during training, the encoder outputs all zeros for the city columns instead of raising an error. The ColumnTransformer then becomes the first step of a Pipeline, with the model as the last step.

Worked example

Predict whether a customer buys a product from age, income and city. To keep the example easy to check, the label follows a simple rule: customers with income above 25 buy.

import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

rng = np.random.default_rng(100)
data = pd.DataFrame({
    "age":    rng.integers(20, 60, 300),
    "income": rng.uniform(5, 50, 300).round(1),
    "city":   rng.choice(["Hanoi", "Ho Chi Minh City", "Hue"], 300),
})
data["bought"] = (data["income"] > 25).astype(int)

x = data[["age", "income", "city"]]
y = data["bought"]
x_train, x_test, y_train, y_test = train_test_split(
    x, y, test_size=0.2, random_state=1009, stratify=y)

pre = ColumnTransformer([
    ("num", StandardScaler(), ["age", "income"]),
    ("cat", OneHotEncoder(handle_unknown="ignore"), ["city"]),
])
pipe = Pipeline([("pre", pre), ("model", RandomForestClassifier(random_state=100))])

scores = cross_val_score(pipe, x_train, y_train, cv=5)   # preprocessing refit in each fold
print(scores.round(3))

pipe.fit(x_train, y_train)
print(round(pipe.score(x_test, y_test), 3))

new_customer = pd.DataFrame({"age": [35], "income": [40.0], "city": ["Da Nang"]})
print(pipe.predict(new_customer))

Expected output: five cross-validation accuracies, all close to 1.0, and a test accuracy close to 1.0, because the label follows a clean income rule. The last line prints [1]: the new customer has income 40, and the unseen city "Da Nang" causes no error thanks to handle_unknown="ignore". At no point did we call fit on test data by hand; the Pipeline handled it.

Common mistakes

  • Standardising before cross-validation. Scaling the whole training set and then running cross_val_score leaks every validation fold; put the scaler inside the Pipeline.
  • Putting the model in the middle. Only the last step of a Pipeline may be the estimator; preprocessing steps come first.
  • Forgetting handle_unknown="ignore". A category unseen in training then raises an error at prediction time.
  • Fitting a separate scaler and also using a Pipeline. Scaling twice gives inconsistent inputs; let the Pipeline own every step.

Summary

  • A Pipeline chains preprocessing steps and a model into one object.
  • fit runs fit_transform on each preprocessing step and fits the model; predict only transforms and predicts.
  • With cross-validation, a Pipeline refits preprocessing inside every fold, preventing leakage.
  • ColumnTransformer applies different transformers to numerical and categorical columns.
  • Data leakage gives no error message, so make the safe structure the default.

Flashcards · Machine Learning Pipelines in Scikit-learn

8 words
WordMeaningExample
  • Pipeline

    scikit-learn object that chains preprocessing steps and a final model so they are fitted and applied in the correct order.

    Pipeline([("scaler", StandardScaler()), ("model", RandomForestClassifier())])

  • pipe.fit()

    Fits every preprocessing step on the training data, transforms it, then trains the final model.

    pipe.fit(x_train, y_train)

  • pipe.predict()

    Transforms new data with the already-fitted steps, then makes predictions; nothing is refitted.

    y_pred = pipe.predict(x_test)

  • cross_val_score()

    Evaluates a model on several train/validation folds and returns one score per fold.

    cross_val_score(pipe, x_train, y_train, cv=5)

  • ColumnTransformer

    Applies different transformers to different groups of columns and joins the results.

    ("num", StandardScaler(), ["age", "income"])

  • handle_unknown="ignore"

    OneHotEncoder option that outputs all zeros for unseen categories instead of raising an error.

    OneHotEncoder(handle_unknown="ignore")

  • Silent data leakage

    Test or validation information influencing preprocessing with no error shown, so scores look better than reality.

    Scaling the whole training set before cross-validation.

  • Fold

    One of the parts the training data is split into during cross-validation.

    cv=5 creates five folds.