Streaming Engines
Last Updated: May 26, 2026
Streaming engines are distributed runtimes for continuous data processing. They filter, join, enrich, aggregate, and route events as data arrives.
Reading messages quickly is only the entry point. Production streaming also needs event-time semantics, state management, checkpoints, replay, backpressure handling, and sink consistency.
In this chapter, you will learn:
- The core concepts behind streaming engines
- How Apache Flink, Kafka Streams, and Spark Structured Streaming differ
- How state, watermarks, checkpoints, and output commits work
- How to choose an engine for a production system