02.06 · Walkthrough
dbt Models for ML Transformations
Create dbt models that materialize staging, intermediate, and feature tables with tests and documentation.
dbt models can turn raw warehouse data into tested ML feature tables by separating staging cleanup, intermediate domain logic, and training-ready outputs. The important design choices are grain, entity keys, timestamps, leakage prevention, materialisation strategy, and documentation that makes feature assumptions clear to both ML and data engineering teams.
What this lesson answers
- how to structure dbt models for ML features
- dbt staging intermediate feature table best practices
- how to prevent data leakage in dbt features
Notes
In dbt, models are select statements that define reusable transformations and materialize them as views or tables in the warehouse. For ML workflows, it is useful to organize models into layers: staging models clean and standardize raw source data, intermediate models join and reshape business entities, and feature models produce training-ready columns at the correct grain. This structure helps engineers separate concerns: staging handles source quirks, intermediate models express domain logic, and feature tables provide stable inputs for model training, batch scoring, or downstream analytics.
Common questions
- How should dbt models be organised for ML feature engineering?
- Use layered models. Staging models should clean and normalise raw sources. Intermediate models should express joins, reshaping, and business rules. Feature models should produce columns at the exact grain needed for training, scoring, or analysis. This separation keeps source quirks away from feature logic and makes ownership easier to reason about.
- What makes a dbt feature table safe for model training?
- A safe feature table makes entity keys, event times, label timing, and aggregation windows explicit. Each feature should be computed only from data that would have existed at prediction time. That design avoids leakage, keeps training and scoring logic aligned, and gives reviewers a concrete way to inspect assumptions in SQL.
- Which dbt tests matter for ML transformation tables?
- Useful tests enforce the shape and trustworthiness of the feature data: non-null keys, unique rows at the declared grain, accepted values for categorical fields, and relationships back to source entities. These tests catch broken joins, duplicate entity-date records, unexpected categories, and missing identifiers before they contaminate training or scoring runs.
Short definition: what is dbt Models for ML Transformations?