Back to the lessonPreparing the printout…

Fundamental of Machine Learning · Module 1: Data Preparation and Model Foundations

Capstone Practice: Planning Preprocessing for a Real Dataset

[markdown] content available only on the platform

[markdown] content available only on the platform

Flashcards · Capstone Practice: Planning Preprocessing for a Real Dataset

Preprocessing plan — A column-by-column table stating each feature's type, how to impute it and how to scale or encode it.

tier: ordinal, most frequent, OrdinalEncoder.

Data audit — Checking a dataset for missing values, duplicates, wrong formats, outliers and low variance before modelling.

df.isna().sum(); df.duplicated().sum()

df.nunique() — Counts distinct values per column; helps spot near-constant columns and categorical features.

houses.nunique()

SimpleImputer — scikit-learn transformer that fills missing values with the mean, median, most frequent value or a constant.

SimpleImputer(strategy="median")

Per-column treatment — Choosing imputation and transformation separately for each feature according to its type.

Median plus StandardScaler for square_feet.

Fit on train only — All preprocessing statistics are learned from training rows and merely applied to other data.

prep.fit_transform(X_train); prep.transform(X_test)

ColumnTransformer — Runs a different preprocessing pipeline on each group of columns and joins the outputs.

("nom", OneHotEncoder(), ["city", "has_garden"])

Target separation — Keeping the target column out of the feature preprocessing so it is never scaled or encoded with inputs.

X = df.drop(columns="price"); y = df["price"]

Fundamental of Machine Learning · Capstone Practice: Planning Preprocessing for a Real Dataset · printed on November 20, 2026