10.02 · Concept
Dataset Versioning Principles
Define a versioning strategy for raw data, transformed data, labels, features, and training splits.
Dataset versioning means treating every ML data artefact as an addressable input to model behaviour: raw snapshots, derived datasets, labels, features, and splits. A useful strategy records provenance, generation logic, parameters, and stable identifiers so regressions, metric changes, and training-serving skew can be investigated against the exact data used.
What this lesson answers
- how should I version ML datasets
- what data artefacts need versioning for reproducibility
- how do I version training validation test splits
Notes
Dataset versioning is the practice of making every data artifact in an ML pipeline identifiable, repeatable, and traceable over time. A working versioning strategy should cover raw data, transformed data, labels, features, and training splits because each of these can change independently and affect model behavior. Raw data should usually be treated as immutable once ingested: instead of overwriting files or tables, create a new snapshot or partition and record when it was captured, where it came from, and any schema or source changes.
Common questions
- Should raw data be overwritten when a source changes?
- No. Raw data should usually be kept immutable after ingestion. When the source changes, create a new snapshot or partition instead of replacing the old one. Record where it came from, when it was captured, and what changed in the source or schema, so past models can still be reproduced.
- What should be recorded for transformed datasets and features?
- Record the raw input version, the code or logic that produced the artefact, relevant parameters, and the runtime context. For features, also capture the computation definition and source snapshot so offline training values can be compared with online serving values when investigating skew.
- Why version train, validation, and test splits separately?
- Splits affect reported metrics even when the underlying dataset is unchanged. Store the exact record identifiers assigned to each split, the sampling rules, any randomness control used, and the dataset version they came from. That makes model comparisons fair and prevents accidental metric changes from regenerated splits.
Short definition: what is Dataset Versioning Principles?