Skip to content

dbt Models for ML Transformations

dbt models for ML transformations are version-controlled SQL select statements that turn raw warehouse data into tested, documented staging, intermediate, and feature datasets. They define how source records are cleaned, joined, aggregated, and materialised so model training, batch scoring, and analysis consume repeatable inputs rather than ad hoc queries.

ML work often fails at the handoff between data preparation and modelling. Feature logic lives in notebooks, source quirks are fixed differently by different people, and training data cannot be rebuilt exactly later. The problem is not just convenience. If entity keys, timestamps, labels, and aggregation windows are unclear, a model may learn from data that would not have existed at prediction time.

A dbt model is a SQL select statement stored in a project and built into the warehouse as a view, table, or incremental relation. For ML, teams usually layer them: staging models normalise raw sources, intermediate models encode joins and business entities, and feature models produce columns at a declared grain. The feature model should make time semantics visible, especially the point in time, lookback window, and label definition.

The trade-off is that dbt gives structure, not automatic correctness. You still have to design leakage-safe SQL, choose materialisations, and decide when recomputation is acceptable. Views keep simple transformations cheap to maintain but can push cost to every query. Tables and incremental models speed repeated use, but add storage, build complexity, and the possibility of stale or incorrectly updated features.

Engineers meet these models in pull requests, scheduled warehouse builds, feature table definitions, and model training pipelines. Tests commonly assert non-null keys, unique rows at the entity-time grain, accepted categorical values, and valid relationships to source entities. Documentation should state what the model represents, which columns are features or labels, and which assumptions downstream ML code relies on.

Common questions

Are dbt models the same thing as a feature store?
No. dbt can create and maintain feature tables, but a feature store usually adds serving APIs, online storage, point-in-time retrieval, discovery, and governance workflows. dbt is strongest at deterministic warehouse transformations. Whether it is enough depends on whether your ML system only needs batch features or also needs low-latency online features.
How do dbt models help prevent data leakage?
They do not prevent leakage by themselves. They help by making feature SQL reviewable, repeatable, and documented. Engineers can encode timestamp filters, lookback windows, and label cutoffs in one shared transformation instead of scattered notebooks. The important part is explicitly computing features only from information available at the prediction time.
Should ML feature models be views, tables, or incremental models?
It depends on cost, freshness, and query pattern. Views suit lightweight staging or rapidly changing definitions. Tables suit expensive joins and aggregations that are reused for training or scoring. Incremental models suit large historical feature sets where only new periods or entities should be processed, but they require careful update logic.