Back to the lessonPreparing the printout…

Fundamental of Machine Learning · Module 1: Data Preparation and Model Foundations

1.11 Lab: Handling Missing Values and Pipelines

[markdown] content available only on the platform

Flashcards · 1.11 Lab: Handling Missing Values and Pipelines

SimpleImputer — scikit-learn class in sklearn.impute that fills missing values using a chosen strategy.

SimpleImputer(strategy="mean").fit(x_train)

strategy="median" — Imputes with the column median, which is not pulled upwards or downwards by outliers.

Values 1, 2, 9: the median is 2, the mean is 4.

strategy="most_frequent" — Imputes with the most common value; works for categorical columns.

["IT", "IT", nan, "Econ"] -> nan becomes "IT"

statistics_ — Fitted imputer attribute holding the fill value learned for each column.

imp.statistics_ -> [4.]

Fit on train only — Learn imputation and scaling statistics from training data; test and new data only get transform.

imp.fit(x_train); imp.transform(x_test)

fit_transform on test (mistake) — Refitting on the test set fills gaps with test data you will not have at deployment time.

Test [nan, 20] becomes [20, 20] instead of [4, 20].

Pipeline (imputer + scaler) — Chains imputation and scaling so both are fitted on train in order and applied consistently.

Pipeline([("imputer", SimpleImputer()), ("scaler", StandardScaler())])

ColumnTransformer — Applies separate pipelines to numerical and categorical columns of one table.

("cat", Pipeline([...OneHotEncoder()]), ["major"])

get_feature_names_out() — Lists the output column names produced by a fitted transformer.

['num__age' 'cat__major_Econ' 'cat__major_IT']

Placeholder missing values — Markers like "?" or "N/A" are not recognised as missing; convert them to NaN first.

df.replace("?", np.nan)

[file] /api/lesson-media/files/9b417aa2-5e9f-4ae3-a9db-2a3b7f03f372-1794890093151-04269fdcb529906d.zip

Fundamental of Machine Learning · 1.11 Lab: Handling Missing Values and Pipelines · printed on November 20, 2026