Change Data Capture (CDC)
What Is Change Data Capture (CDC)?
Bane Tech Engineering · 8 min read
Change Data Capture (CDC) is a set of techniques for identifying the rows that have been inserted, updated or deleted in a database and delivering those changes to other systems. Instead of repeatedly copying entire tables, CDC moves only what changed — usually shortly after it changed — so downstream systems stay current with far less load on the source.
Why CDC exists
Traditional data movement relies on scheduled batch jobs: every night, export a table and load it somewhere else. This approach is simple, but it has three costs. Data is stale between runs, full extracts put heavy load on production databases, and deletions are easy to miss.
CDC addresses all three by treating the database as a stream of change events rather than a snapshot.
How CDC detects changes
There are four common techniques. Each makes a different trade-off between accuracy, latency and impact on the source database.
1. Timestamp or version columns
Queries select rows whose updated_at column is newer than the last run. Easy to implement, but it requires a reliable column on every table, cannot detect hard deletes and still polls the database.
2. Database triggers
Triggers write every insert, update and delete to a shadow change table. Changes are captured accurately, including deletes, but each write now performs extra work inside the transaction, which can affect performance on busy systems.
3. Snapshot comparison
Two full copies of a table are compared to compute differences. It works on any source but is expensive and only suitable for small tables or infrequent runs.
4. Transaction log reading
Most relational databases record every committed change in a transaction log (for example, the PostgreSQL WAL, the MySQL binlog or the SQL Server transaction log). Log-based CDC reads this log and converts entries into change events. It captures all changes in commit order with minimal impact on the application, but depends on database-specific log access and configuration.
| Technique | Captures deletes | Source impact | Latency |
|---|---|---|---|
| Timestamp columns | No | Medium (polling) | Minutes |
| Triggers | Yes | Higher (extra writes) | Seconds |
| Snapshot comparison | Yes | High | Hours |
| Transaction log | Yes | Low | Sub-second to seconds |
Anatomy of a CDC pipeline
A change event usually contains the operation type (insert, update, delete), the table, the primary key, the new values and often the previous values, plus a position marker so the pipeline can resume exactly where it stopped after a restart.
CDC vs. replication vs. synchronisation
These terms overlap and are often used interchangeably, but they describe different things:
- Database replication maintains a copy of a database, usually of the same engine, for availability or read scaling. It is typically managed by the database itself.
- CDC captures changes as events that can be delivered to any kind of target — a different database, a data warehouse, a message broker, a search index or an application.
- Data synchronisation is the broader goal of keeping two or more systems consistent. CDC is one of the mechanisms used to achieve it.
Common use cases
- Feeding data warehouses and analytics platforms without nightly batch windows.
- Keeping caches and search indexes consistent with the system of record.
- Migrating databases with minimal downtime by streaming changes during cut-over.
- Publishing business events (for example, “order created”) from legacy applications that cannot be modified.
- Triggering processes in an enterprise integration platform when data changes.
Things to plan for
- Schema changes — how the pipeline handles added, removed or renamed columns.
- Ordering and delivery guarantees — whether consumers can tolerate duplicates or out-of-order events.
- Initial load — most pipelines need a consistent snapshot before streaming begins.
- Monitoring — lag between source and target is the key health indicator. See Observability vs. Monitoring.