Is CDC the same as event sourcing?

No. CDC observes changes made to a database after the application has already chosen its storage model. Event sourcing makes the event log the primary source of truth and derives state from it. CDC can produce event-like records, but they usually describe row mutations, not domain events with business intent.

Why use the transaction log instead of triggers or updated_at columns?

The transaction log is the database’s own record of committed changes, so it sees inserts, updates, and deletes consistently and usually with less application overhead. Triggers add write-path logic and can be missed or misconfigured. Timestamp polling often misses deletes, struggles with clock and precision issues, and still requires repeated queries.

How should consumers handle duplicate CDC events?

Assume duplicates can happen unless your whole pipeline proves otherwise. Use the source primary key plus a version, log position, or commit timestamp to make writes idempotent. For materialised views, upsert newer records and ignore older ones. For deletes, ensure the delete marker advances state rather than being treated as an empty update.

Do you still need an initial load when using CDC?

Usually yes. CDC captures changes from a point in the source log, but a downstream system also needs the existing rows. A common design takes an initial snapshot, records the log position it corresponds to, then streams later changes from that position. Getting this handoff right prevents gaps and double application.

Change Data Capture

Change Data Capture is a data integration pattern that records row-level inserts, updates, and deletes from a source database and emits them to another system, usually a stream. It lets downstream stores, indexes, caches, and services stay aligned with operational data without repeatedly scanning whole tables or waiting for batch exports.

CDC exists because databases are often the system of record, while many other systems need a near-current copy of their state. Polling tables is wasteful, misses hard cases such as deletes, and can add load to production queries. Batch exports are simpler but introduce delay and awkward reconciliation. CDC turns database mutation itself into a stream of facts that other systems can consume independently.

The most robust form reads the database’s transaction log, such as a write-ahead log or binlog. The database already writes committed changes there for recovery and replication, so a CDC connector can follow that log and translate entries into events. A typical event names the table, operation, key, changed columns, commit time, and sometimes before and after row images, then publishes it to a broker or downstream sink.

The trade-off is that CDC moves correctness problems into the pipeline. Delivery is commonly at least once, so consumers must tolerate duplicates using keys, versions, timestamps, or idempotent writes. Ordering may only be guaranteed within a partition or key. Deletes need explicit handling, often as tombstones. Schema changes, connector lag, initial snapshots, and log retention all become operational concerns rather than details you can ignore.

Engineers meet CDC in Kafka connectors, database replication slots, managed streaming ingestion tools, data lake table maintenance, search indexing, cache synchronisation, and service integration. It is commonly misunderstood as just “database events”, but the hard part is not emitting messages. The hard part is preserving enough ordering, identity, schema, and delete semantics that downstream state remains a faithful projection of the source.

Go deeper

Everything DataChange Data CaptureUse a CDC pattern to capture database inserts, updates, and deletes and publish them into a streaming pipeline.Open the lesson

Common questions

Is CDC the same as event sourcing?: No. CDC observes changes made to a database after the application has already chosen its storage model. Event sourcing makes the event log the primary source of truth and derives state from it. CDC can produce event-like records, but they usually describe row mutations, not domain events with business intent.
Why use the transaction log instead of triggers or updated_at columns?: The transaction log is the database’s own record of committed changes, so it sees inserts, updates, and deletes consistently and usually with less application overhead. Triggers add write-path logic and can be missed or misconfigured. Timestamp polling often misses deletes, struggles with clock and precision issues, and still requires repeated queries.
How should consumers handle duplicate CDC events?: Assume duplicates can happen unless your whole pipeline proves otherwise. Use the source primary key plus a version, log position, or commit timestamp to make writes idempotent. For materialised views, upsert newer records and ignore older ones. For deletes, ensure the delete marker advances state rather than being treated as an empty update.
Do you still need an initial load when using CDC?: Usually yes. CDC captures changes from a point in the source log, but a downstream system also needs the existing rows. A common design takes an initial snapshot, records the log position it corresponds to, then streams later changes from that position. Getting this handoff right prevents gaps and double application.

Every term