08.05 · Concept
Label and Concept Drift
Distinguish feature drift, label drift, and concept drift and select monitoring signals for each.
No video curated for this lesson yet
This lesson is written, ordered and part of the path - the video slot is the only thing still open. We are working through Everything Data lesson by lesson; 55 of 85 have their video so far.
The written notes below cover this idea in full - you lose nothing by reading instead of watching.
Feature drift changes the inputs, label drift changes the outcomes, and concept drift changes the mapping between them. Monitoring should match the failure mode: input distributions for early warning, delayed ground truth for outcome shifts, and performance or error by segment for changes in model behaviour.
What this lesson answers
- feature drift versus label drift versus concept drift
- what signals monitor concept drift in production
- how to detect label drift with delayed labels
Notes
In production data systems, drift means the relationship between the data your model or downstream logic was built on and the data it sees now has changed. Feature drift is a change in the input data distribution: for example, customer ages, transaction amounts, device types, or missing-value rates start looking different from training or baseline data. This can happen because user behavior changes, a new market is added, an upstream schema changes, or instrumentation breaks.
Common questions
- What is the difference between feature drift and concept drift?
- Feature drift means the inputs have changed compared with a baseline, such as different category frequencies, missing values, ranges, or vector similarity. Concept drift means the relationship between those inputs and the outcome has changed, so the same input pattern now implies a different result. Feature drift can be seen before labels arrive; concept drift usually needs outcome data.
- How should label drift be monitored in production systems?
- Monitor the distribution and availability of ground-truth outcomes. Useful checks include class balance, positive rate, outcome averages, label delay, missing labels, and differences between expected and observed results. Separate genuine outcome movement from data collection problems, because late, dropped, or redefined labels can look like label drift.
- Which drift type matters most for model performance?
- Concept drift is usually the most direct threat to model performance because it means the learned relationship no longer holds. Feature drift may be harmless if the model still generalises, and label drift may reflect a broad outcome shift. Performance metrics, residuals, calibration, and segment-level errors are the strongest signals for concept drift.
Short definition: what is Label and Concept Drift?