Schema Registry and Event Contracts
A schema registry is a central service that stores versioned definitions of event payloads and enforces rules for how those definitions may change. Together with event contracts, it lets producers and consumers agree on field names, types, optionality and meaning, so streaming data can evolve without silently breaking downstream code.
Streaming systems fail in awkward ways when messages are treated as unstructured blobs. A producer can rename a field, change a type, or stop sending a value while consumers are still deployed with old assumptions. Because producers and consumers usually release independently, there is rarely a single safe cutover moment. The schema registry exists to make the event shape explicit and checkable, instead of relying on tribal knowledge, examples, or whatever the latest producer happens to emit.
In practice, a producer serialises an event using a registered schema, such as an Avro, Protobuf, or JSON Schema definition. The event carries, or can be associated with, a schema identity. A consumer uses that identity to fetch the right definition and decode the bytes into expected fields and types. When someone proposes a new schema version, the registry compares it with earlier versions and accepts or rejects it according to the configured compatibility mode.
The trade-off is that event evolution becomes more deliberate. Engineers must define optionality, defaults, names and types carefully, and some quick changes are blocked because they would strand existing consumers. Compatibility is also not magic: a schema can prove that a field exists and has a type, but it cannot fully prove that the business meaning stayed the same. Misunderstanding this is common. Contracts reduce breakage, but they do not remove the need for migration plans.
Engineers meet schema registries around Kafka and other streaming platforms, usually in producer libraries, consumer deserialisers, CI/CD checks, and deployment gates. A typical workflow is to change the schema first, run compatibility validation, release producers that write both old and new information if needed, then migrate consumers before removing anything. Treat the event as a public API for other services, not as a private object dumped onto a topic.
Common questions
- Is a schema registry just documentation for events?
- No. Documentation explains intent to humans, while a schema registry is used by software at runtime or deployment time. It stores versioned schemas, gives them identities, and can reject incompatible changes. Good descriptions still matter, because field semantics are part of the contract, but the registry makes the structural part enforceable.
- What counts as a breaking change to an event contract?
- It depends on the compatibility rule and serialisation format, but common breaking changes include removing a required field, changing a field type, or renaming a field without a transition. Adding an optional field with a safe default is usually compatible. The key question is whether old and new producers and consumers can coexist safely.
- Do event contracts mean consumers never need to change?
- No. They mean consumers get predictable evolution instead of surprise failures. Consumers still need to adopt new fields, retire old logic, and handle migrations. The registry gives teams a controlled path: add before remove, support overlap where needed, and let automated checks catch changes that would break deployed readers.