Period 15 · Review: The Full Workflow
[video] Lecture video — Period 15: Review: The Full Workflow — /api/lesson-media/videos/4d3653ff-a3ca-4f63-bf8e-bb93c553e599-1794847916692-052632a70be3ad87.mp4
[markdown] content available only on the platform
[markdown] content available only on the platform
Flashcards · Review: The Full Workflow
Full cleaning workflow — Keep the raw copy, explore, clean and transform with reasons, rerun and check, then report findings and limits.
Raw file to notebook to clean data, quality log and report.
Known total — A total calculated only from records with complete information, reported with its coverage.
180 from 2 of 3 sales.
Coverage — The share of records that a result actually includes.
`sales["revenue"].notna().sum()` gives 2 of 3.
Assumption effect — The change in a result caused only by an assumed value, which should be stated.
Assuming 0% discount adds 100 to the total.
Sensitivity check — Testing whether a different reasonable treatment changes the conclusion.
Compare totals with rows dropped versus filled.
Unresolved record — A record kept visible in a review table or log because evidence to fix it is missing.
A sale with an unknown discount awaiting verification.
Before-and-after comparison — Comparing row counts, missing rates and key totals around each important treatment.
Rows 108 to 100; revenue total before and after.
sum() skips missing — pandas `sum()` ignores missing values by default, so a total can be silently partial.
100 + NaN + 80 sums to 180.
[pdf] Lecture slides — Period 15 (PDF) — /api/lesson-media/pdfs/4d3653ff-a3ca-4f63-bf8e-bb93c553e599-1794873740465-e0a7ee754d48a48b.pdf
