You are viewing a public preview
Sign in to save your progress, do assignments, join discussions and get feedback.
When working manually, you must remember the exact order and the exact rules: fit the scaler on train, transform test, and only then train the model. Forgetting just once leaks data without any error message — the model still runs, the results still look great, they are just wrong.
Pipeline combines all the steps into a single object. Call fit once, and the library automatically applies the correct rule to each step.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
pipe = Pipeline([
("scaler", StandardScaler()),
("model", RandomForestClassifier(random_state=100)),
])
pipe.fit(x_train, y_train) # scaler.fit_transform + model.fit
y_pred = pipe.predict(x_test) # scaler.transform + model.predict
When used together with cross-validation, the Pipeline refits the scaler separately for each fold. When working manually, people often standardize once on the entire training set and only then split into folds — and so every validation fold has already been leaked into.
from sklearn.model_selection import cross_val_score
scores = cross_val_score(pipe, x_train, y_train, cv=5) # safe
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
pre = ColumnTransformer([
("num", StandardScaler(), ["age", "income"]),
("cat", OneHotEncoder(handle_unknown="ignore"), ["city"]),
])
pipe = Pipeline([("pre", pre), ("model", RandomForestClassifier())])
Standardizing the data before passing it into cross-validation. The results will look better than reality and you will not notice, because nothing reports an error. Putting the scaler inside the Pipeline is the sure way to avoid it.
By the end of this lesson you can:
Pipeline solves.Pipeline that chains a scaler and a model, and call fit and predict on it.Pipeline with cross_val_score without leaking validation data.ColumnTransformer to treat numerical and categorical columns differently.Key idea: A
Pipelinepacks preprocessing and the model into one object, so every step automatically follows the rule "fit on training data, only transform new data". This prevents silent data leakage, especially inside cross-validation.
Working by hand, you must remember the exact order and the exact rules: fit the scaler on train, transform test, then train the model. Forget once and data leaks without any error message. The model still runs and the results still look great; they are simply wrong.
A Pipeline combines the steps into a single object. You call fit once and the library applies the correct rule at each step:
| You call | What the Pipeline does |
|---|---|
pipe.fit(x_train, y_train) | scaler.fit_transform(x_train), then model.fit(...) on the result |
pipe.predict(x_test) | scaler.transform(x_test), then model.predict(...) |
A typical definition lists named steps in order: Pipeline([("scaler", StandardScaler()), ("model", RandomForestClassifier(random_state=100))]). All steps except the last must be transformers; the last step is the model.
Cross-validation splits the training set into several folds and, in turn, uses each fold for validation. When a Pipeline is passed to cross_val_score(pipe, x_train, y_train, cv=5), the scaler is refit separately inside each fold, using only that fold's training part. Working by hand, people often standardise the whole training set once and only then split into folds. Every validation fold has then already influenced the scaling, and the scores look better than they really are.
Real tables mix column types. ColumnTransformer applies a different transformer to each group of columns and joins the results side by side:
("num", StandardScaler(), ["age", "income"]) scales the numerical columns;("cat", OneHotEncoder(handle_unknown="ignore"), ["city"]) one-hot encodes the city column.handle_unknown="ignore" matters in production: if a new customer comes from a city never seen during training, the encoder outputs all zeros for the city columns instead of raising an error. The ColumnTransformer then becomes the first step of a Pipeline, with the model as the last step.
Predict whether a customer buys a product from age, income and city. To keep the example easy to check, the label follows a simple rule: customers with income above 25 buy.
import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
rng = np.random.default_rng(100)
data = pd.DataFrame({
"age": rng.integers(20, 60, 300),
"income": rng.uniform(5, 50, 300).round(1),
"city": rng.choice(["Hanoi", "Ho Chi Minh City", "Hue"], 300),
})
data["bought"] = (data["income"] > 25).astype(int)
x = data[["age", "income", "city"]]
y = data["bought"]
x_train, x_test, y_train, y_test = train_test_split(
x, y, test_size=0.2, random_state=1009, stratify=y)
pre = ColumnTransformer([
("num", StandardScaler(), ["age", "income"]),
("cat", OneHotEncoder(handle_unknown="ignore"), ["city"]),
])
pipe = Pipeline([("pre", pre), ("model", RandomForestClassifier(random_state=100))])
scores = cross_val_score(pipe, x_train, y_train, cv=5) # preprocessing refit in each fold
print(scores.round(3))
pipe.fit(x_train, y_train)
print(round(pipe.score(x_test, y_test), 3))
new_customer = pd.DataFrame({"age": [35], "income": [40.0], "city": ["Da Nang"]})
print(pipe.predict(new_customer))
Expected output: five cross-validation accuracies, all close to 1.0, and a test accuracy close to 1.0, because the label follows a clean income rule. The last line prints [1]: the new customer has income 40, and the unseen city "Da Nang" causes no error thanks to handle_unknown="ignore". At no point did we call fit on test data by hand; the Pipeline handled it.
cross_val_score leaks every validation fold; put the scaler inside the Pipeline.handle_unknown="ignore". A category unseen in training then raises an error at prediction time.fit runs fit_transform on each preprocessing step and fits the model; predict only transforms and predicts.ColumnTransformer applies different transformers to numerical and categorical columns.Flashcards · Machine Learning Pipelines in Scikit-learn
8 wordsscikit-learn object that chains preprocessing steps and a final model so they are fitted and applied in the correct order.
Fits every preprocessing step on the training data, transforms it, then trains the final model.
Transforms new data with the already-fitted steps, then makes predictions; nothing is refitted.
Evaluates a model on several train/validation folds and returns one score per fold.
Applies different transformers to different groups of columns and joins the results.
OneHotEncoder option that outputs all zeros for unseen categories instead of raising an error.
Test or validation information influencing preprocessing with no error shown, so scores look better than reality.
One of the parts the training data is split into during cross-validation.