1.10 Machine Learning Pipeline
[video] /api/lesson-media/videos/9b417aa2-5e9f-4ae3-a9db-2a3b7f03f372-1794891204736-dabecb09d1f0d20a.mp4
[markdown] content available only on the platform
[markdown] content available only on the platform
Flashcards · 1.10 Machine Learning Pipeline
Machine learning pipeline — The ordered sequence of steps in an ML project, from data collection to model deployment.
Nine steps in the lecture's diagram.
Data collection — Gathering raw data from sources such as databases, APIs, sensors or surveys.
Exporting past sales from a company database.
Statistics (pipeline step) — Basic statistical analysis to understand the data's structure before cleaning it.
Mean, median and range of price.
Data preprocessing — Cleaning and transforming data: missing values, duplicates, formats, outliers, scaling and encoding.
Impute square_feet, one-hot encode city.
Data visualization — Plotting data to explore relationships and reveal patterns or problems numbers may hide.
A scatter plot of price against area.
Model selection — Choosing which algorithms to try based on the problem type, the data and the goal.
Numeric output, so shortlist regression models.
Model validation — Checking and tuning model performance on the validation set; it sits between training and testing.
Compare three candidate models on validation data.
Model testing — Measuring final performance once on the held-out test set for an honest estimate.
Report the test error before launch.
Model deployment — Putting the finished, tested model into production so it makes real predictions.
Suggesting prices on a listing website.
