Module 05
Streaming & Real-Time
Design and implement streaming data systems for low-latency features, real-time analytics, and event-driven ML applications.
7 lessons · 8 videos · 1h 27m- 05.01
Streaming System Concepts
Explain events, topics, partitions, offsets, watermarks, event time, processing time, and exactly-once semantics.
- 05.02
Kafka Producers, Consumers, and Topics
Create a Kafka topic, publish structured events, consume them, and inspect offsets and consumer group behavior.
- 05.03
Schema Registry and Event Contracts
Define an event schema with compatibility rules that prevents breaking downstream streaming consumers.
- 05.04
Flink Stream Processing
Build a Flink job that computes windowed aggregates with event-time handling and late-arriving data support.
- 05.05
Streaming Features for ML
Design streaming feature computations for counters, rolling windows, session attributes, and real-time personalization.
- 05.06
Change Data Capture
Use a CDC pattern to capture database inserts, updates, and deletes and publish them into a streaming pipeline.
- 05.07
Backpressure, Replay, and Failure Recovery
Diagnose lag, backpressure, poison messages, and replay requirements in a streaming architecture.