Back to the lessonPreparing the printout…

Data Cleaning and Preprocessing with Python · Module 3 — Prepare, Report and Reproduce

Period 13 · Building a Rerunnable Cleaning Workflow

[video] Lecture video — Period 13: Building a Rerunnable Cleaning Workflow — /api/lesson-media/videos/4d3653ff-a3ca-4f63-bf8e-bb93c553e599-1794846243337-b725847b1eb48e00.mp4

[markdown] content available only on the platform

[markdown] content available only on the platform

Flashcards · Building a Rerunnable Cleaning Workflow

Rerunnable notebook — A notebook that starts from the raw data and gives the same results whenever input, rules and software are unchanged.

Restart kernel, Run All, compare outputs.

Hidden state — Variables left in memory from cells that were run earlier or later deleted, which make results hard to repeat.

A `df` defined in a cell you removed.

Cleaning function — A function with a narrow purpose that takes a DataFrame, works on a copy and returns a result.

`def convert_numeric(frame): out = frame.copy(); ...; return out`

assert — A statement that stops execution with AssertionError when a required condition is false.

`assert clean["transaction_id"].is_unique, "IDs must be unique"`

Processing log — A table recording each step and how many rows it removed, sent to review or accepted.

6 input = 1 duplicate + 2 review + 3 accepted

pd.testing.assert_frame_equal() — Raises an error if two DataFrames differ, used to confirm a rerun gives the same result.

`assert_frame_equal(key(clean), key(clean2))`

index=False — Option of `to_csv` that stops pandas writing the row index as an extra column.

`clean.to_csv("output/sales_clean.csv", index=False)`

Read-back check — Reloading an exported file to confirm it exists, has the right shape and keeps values correctly.

`pd.read_csv(path, dtype={"transaction_id": str})`

Required field — A column that must be filled for a record to be accepted, unlike optional fields.

`unit_price` is required; a missing price sends the row to review.

[pdf] Lecture slides — Period 13 (PDF) — /api/lesson-media/pdfs/4d3653ff-a3ca-4f63-bf8e-bb93c553e599-1794873731926-2ce4f2efb60004cb.pdf

[markdown] content available only on the platform

Data Cleaning and Preprocessing with Python · Period 13 · Building a Rerunnable Cleaning Workflow · printed on November 20, 2026