How are contract tests different from ordinary data quality tests?

Data quality tests ask whether a dataset is acceptable in general; contract tests ask whether a producer still honours specific promises made to specific consumers. A freshness check, uniqueness rule, or enum validation can be either. It becomes a contract test when it represents a consumer dependency and gates a producer or version change.

Are schema checks enough for a pipeline contract?

No. This is a common misunderstanding. Schema compatibility only says the shape still looks usable. A producer can keep the same columns and types while changing units, accepted values, deduplication behaviour, or metric definitions. Useful contracts include semantic assertions for the business meaning that consumers rely on, not just structural checks.

Who should own contract tests, the producer or the consumer?

Ownership should be shared, but responsibilities differ. Consumers define the assumptions that would break them, and producers run those checks before releasing changes. In practice, the tests often live near producer pipeline code so they gate deployment, while review and versioning involve the affected consumers.

What happens when a breaking change is intentional?

The test should fail, because the old contract is no longer being honoured. The fix is not to delete the test silently. Create a new contract version, release the producer against it, migrate consumers that can handle the new behaviour, and retire the old contract once dependent systems have moved.

Contract Tests for Pipelines

Contract tests for pipelines are executable checks that verify a data producer still satisfies the agreed expectations of its consumers before a pipeline change is released. They cover both shape and meaning: columns, types, nullability, valid values, uniqueness, freshness, and business rules such as how a metric or status field is interpreted.

Data pipelines break quietly because their interfaces are often implicit. A producer can rename a field, widen a type, alter an enum, delay delivery, or change the meaning of a measure while downstream jobs still run. The result is worse than a crash: dashboards, features, and reports may be wrong but look healthy. Contract tests exist to move those assumptions out of chat threads and incident notes into code that fails before deployment.

A contract is written as a set of assertions over a producer’s output from representative data, a staging run, or another controlled boundary. Some assertions are structural: required columns exist, types match, nulls are allowed only where agreed, and keys remain unique. Others are semantic: totals are not negative, timestamps are plausible, currencies come from an accepted set, and a status value has the meaning consumers depend on. The pipeline change is rejected when those checks fail.

The trade-off is that contracts need ownership and maintenance. Overly weak contracts miss real breakages; overly strict contracts block harmless evolution. The honest answer is that the right strictness depends on consumer risk, data volatility, and release cadence. Contract tests also do not prove all production data is correct. They check agreed compatibility at a boundary, so they should complement, not replace, runtime monitoring and data quality checks.

Engineers usually meet these tests in CI/CD, staging validation, data build tools, schema registries, orchestration checks, or consumer-driven test suites. A producer runs the contracts required by downstream jobs, dashboards, machine learning features, or APIs before shipping. A consumer may also test a new upstream version before accepting it. When a breaking change is intentional, teams version the contract, migrate consumers, and retire the old behaviour deliberately.

Go deeper

Everything DataContract Tests for PipelinesWrite producer-consumer data contract tests that catch breaking schema and semantic changes before deployment.Open the lesson

Common questions

How are contract tests different from ordinary data quality tests?: Data quality tests ask whether a dataset is acceptable in general; contract tests ask whether a producer still honours specific promises made to specific consumers. A freshness check, uniqueness rule, or enum validation can be either. It becomes a contract test when it represents a consumer dependency and gates a producer or version change.
Are schema checks enough for a pipeline contract?: No. This is a common misunderstanding. Schema compatibility only says the shape still looks usable. A producer can keep the same columns and types while changing units, accepted values, deduplication behaviour, or metric definitions. Useful contracts include semantic assertions for the business meaning that consumers rely on, not just structural checks.
Who should own contract tests, the producer or the consumer?: Ownership should be shared, but responsibilities differ. Consumers define the assumptions that would break them, and producers run those checks before releasing changes. In practice, the tests often live near producer pipeline code so they gate deployment, while review and versioning involve the affected consumers.
What happens when a breaking change is intentional?: The test should fail, because the old contract is no longer being honoured. The fix is not to delete the test silently. Create a new contract version, release the producer against it, migrate consumers that can handle the new behaviour, and retire the old contract once dependent systems have moved.

Every term