What metadata should an ML experiment record?

Record the parameters used before training, the metrics produced during evaluation, the artifacts generated by the run, the dataset version, the model version, and the execution environment. Environment metadata should include dependency versions, container image, Python version, hardware type, random seed, and the relevant Git commit.

Why is logging metrics not enough for reproducibility?

Metrics say how a run performed, but not why it performed that way. To reproduce or explain a result, you need the exact data snapshot, code state, configuration, dependencies, runtime conditions, and produced artifacts. Without that context, a better score cannot be reliably compared, audited, or recreated.

Where should experiment tracking happen in an ML workflow?

Experiment tracking should be built into the training pipeline, not added manually after a notebook run. Automatic logging gives engineers a searchable history of runs, supports comparisons across experiments, and preserves the evidence needed to promote, roll back, or audit a model in production.

Data Versioning & ML Reproducibility

10.05 · Walkthrough

Experiment Tracking Metadata

Log parameters, metrics, artifacts, dataset versions, model versions, and environment metadata for an ML experiment.

No video curated for this lesson yet

This lesson is written, ordered and part of the path - the video slot is the only thing still open. We are working through Everything Data lesson by lesson; 55 of 85 have their video so far.

The written notes below cover this idea in full - you lose nothing by reading instead of watching.

Experiment tracking metadata is the run record that ties an ML result to its configuration, data, code, artifacts, model output, and runtime environment. Capturing it automatically makes experiments comparable, reproducible, auditable, and useful for deployment decisions when notebooks, source data, dependencies, or feature pipelines have changed.

What this lesson answers

what metadata should I log for ML experiments
why are metrics alone insufficient for experiment tracking
how to make ML experiment runs reproducible

Notes

Experiment tracking metadata is the operational record of what happened during a machine learning run. For a working engineer, the goal is to make every experiment explainable, comparable, and repeatable after the fact. At minimum, each run should capture the input parameters, resulting metrics, produced artifacts, dataset version, model version, and execution environment. Parameters describe what was configured before training, such as learning rate, feature set, batch size, or preprocessing options.

Common questions

What metadata should an ML experiment record?: Record the parameters used before training, the metrics produced during evaluation, the artifacts generated by the run, the dataset version, the model version, and the execution environment. Environment metadata should include dependency versions, container image, Python version, hardware type, random seed, and the relevant Git commit.
Why is logging metrics not enough for reproducibility?: Metrics say how a run performed, but not why it performed that way. To reproduce or explain a result, you need the exact data snapshot, code state, configuration, dependencies, runtime conditions, and produced artifacts. Without that context, a better score cannot be reliably compared, audited, or recreated.
Where should experiment tracking happen in an ML workflow?: Experiment tracking should be built into the training pipeline, not added manually after a notebook run. Automatic logging gives engineers a searchable history of runs, supports comparisons across experiments, and preserves the evidence needed to promote, roll back, or audit a model in production.

Short definition: what is Experiment Tracking Metadata?

10.04 · Data Versioning & ML ReproducibilityLakeFS for Data Lake Branches 10.06 · Data Versioning & ML ReproducibilityTraining and Serving Data Lineage