3.P4 Standardization with StandardScaler
[video] /api/lesson-media/videos/9b417aa2-5e9f-4ae3-a9db-2a3b7f03f372-1794878470521-f37f1efea8fa6d0c.mp4
[markdown] content available only on the platform
[markdown] content available only on the platform
Flashcards · 3.P4 Standardization with StandardScaler
Standardisation (z-score) — Rescaling a value as z = (x - u) / s so the column has mean 0 and standard deviation 1.
Glucose 140 with u = 120, s = 16.33 -> 1.225
StandardScaler — scikit-learn transformer that standardises each column using its mean and standard deviation.
scaler = StandardScaler()
fit() — Learns the statistics (mean and standard deviation of each column) without changing the data.
scaler.fit(x_train)
transform() — Applies already-learned statistics to data; used alone on the test set.
x_test = scaler.transform(x_test)
fit_transform() — Fits on the data and transforms it in one call; used on the training set.
x_train = scaler.fit_transform(x_train)
Data leakage (scaling) — Letting test-set statistics influence preprocessing, which makes evaluation look better than reality.
Calling fit_transform on x_test.
scaler.mean_ / scaler.scale_ — Attributes holding the per-column mean and standard deviation learned by fit.
scaler.mean_ -> [120.]
Feature range imbalance — Features measured on very different scales, which can make a model over-weight large-valued features.
Pregnancies 0-17 vs Glucose up to about 200.
