07.03 · Walkthrough
Dagster Software-Defined Assets
Model a pipeline in Dagster as assets with dependencies, partitions, metadata, and materialization checks.
Dagster software-defined assets model data pipelines around the datasets and artefacts being maintained, rather than only around ordered tasks. Assets declare dependencies, partitions, metadata, and checks in code, giving Dagster enough structure to compute lineage, rerun affected work, expose operational context, and attach data quality to each materialisation.
What this lesson answers
- how do Dagster software defined assets work
- how to model asset dependencies in Dagster
- what are Dagster materialisation checks for
Notes
Dagster’s software-defined assets shift the focus of orchestration from “run these tasks in this order” to “produce and maintain these data objects.” An asset represents something meaningful in the data platform, such as a table, file, feature set, dashboard input, or model artifact. Instead of defining a pipeline only as a sequence of operations, you declare the assets your system owns and the dependencies between them. Dagster then uses that asset graph to understand what needs to be materialized, what is upstream or downstream of a change, and how failures affect the rest of the system.
Common questions
- What is a software-defined asset in Dagster?
- A software-defined asset is a data object described in code, such as a table, file, model output, or dashboard input. The definition says how the object is produced and what other assets it depends on. Dagster uses those definitions to build an asset graph, track materialisations, and reason about what is current or stale.
- How are partitions used with Dagster assets?
- Partitions describe repeated slices of an asset, commonly by time or another business dimension. Instead of treating a table or output as one indivisible unit, Dagster can track which slices exist, which failed, and which are affected by upstream changes. That makes backfills and targeted recomputation much more precise.
- Why put metadata and checks on asset materialisations?
- Metadata records useful facts about a produced asset, such as counts, paths, schemas, links, or quality summaries. Checks express conditions the asset should satisfy after production. Keeping both with the materialisation gives operators a direct view of what was created, whether it looks valid, and where to investigate when something breaks.
Short definition: what is Dagster Software-Defined Assets?