S/DMORROW
WORKSHOP
DATA ENGINEERING / STARTER 01ONE-TIME DOWNLOAD · £5

Keep the history. Test the boundaries.

Understand changes without rewriting the past.

An original SCD Type 2 learning kit for analysts and junior data engineers. Run a Python/SQLite reference, inspect nine regression tests and work through a design checklist before adapting it to your warehouse.

View the £5 kit on Gumroad →

Six files. A precise scope.

Runnable Python reference, nine tests, setup guide, design workbook, primary-source references and a use/adaptation licence. Python 3.11+, no third-party packages. The demo uses synthetic data in memory.

Tested: half-open interval boundaries, unchanged snapshots, duplicate inputs, late-event rejection, timezone equivalence and rollback after an insert fails.

Not included: PostgreSQL, Snowflake, BigQuery or Databricks integrations; production deployment; concurrent loaders; late-history backfills; privacy-erasure workflows; ongoing support. No performance or commercial outcomes are promised.

Free design preview

Before you preserve history, decide what history means.

  1. Define the durable business key and tracked attributes.
  2. Choose event time or ingestion time explicitly.
  3. Use start-inclusive, end-exclusive intervals and test the exact change boundary.
  4. Choose an unknown-member or quarantine policy for facts before known history.
  5. Design late-event, deletion and concurrent-loader handling separately.
  6. Run integration tests on your actual warehouse before production.

The full download provides the executable example and tests behind this checklist. Its implementation is SQLite only.

Concept references: Kimball Group Type 2 · SQLite transactions. No affiliation or endorsement.