A green paper broom clearing loose paper fragments from a spreadsheet.All guides

Data cleanup

Find hidden differences and bring mismatched exports back into order.

Keep a copy of the source, make one change, and compare the result.

Two paper tables with added, removed and changed tiles.
Data cleanup3 min read

Compare CSV snapshots by ID

Compare two keyed CSV snapshots in Python and separate new IDs, missing IDs, and changed field values without relying on row order.

Example files included
Two paper keys connect matching pairs of table cells.
Data cleanup3 min read

Find CSV duplicates using a two-column key

Check duplicate pairs such as order and line number with tuple keys, without merging distinct records through string concatenation.

Example files included
Paired paper ID tags leave one tag without a match.
Data cleanup3 min read

Find CSV IDs missing from a reference list

Use an exact ID comparison to find records absent from a reference CSV, preserve leading zeros, and keep unmatched rows for review.

Example files included
A green pointer marks a gap between paper stepping stones.
Data cleanup3 min read

Find gaps in an expected CSV number sequence

Check a bounded expected integer sequence against a CSV export, report gaps and duplicates separately, and avoid inferring a range from incomplete data.

Example files included
A paper balance weighs input records against kept and rejected stacks.
Data cleanup3 min read

Reconcile CSV counts after filtering

Account for accepted, filtered, and invalid CSV records in Python so a cleanup step cannot silently lose rows between input and output.

Example files included
Green paper tweezers lift empty row strips from a spreadsheet while populated rows stay in place.
Data cleanup6 min read

Remove blank CSV rows with Python

Remove empty CSV records with Python while preserving quoted line breaks, partial rows, zeros, and spaces. Includes a tested script and sample file.

Example files included
A wide fan of paper columns folds into a long record strip.
Data cleanup3 min read

Unpivot CSV month columns into a long table

Turn selected CSV columns into period-and-value rows in Python while preserving blank observations and repeating the identifier deliberately.

Example files included