Lambda Architecture solves the latency-accuracy trade-off by running batch and stream processing in parallel. It works, but maintaining two separate codebases for the same logic is painful. Every change requires updates to both systems. Testing is complex. Bugs can cause the two implementations to diverge.

In 2014, Jay Kreps (co-creator of Apache Kafka) proposed a simpler alternative: Kappa Architecture. The core idea is radical. Instead of running batch and stream in parallel, use only stream processing. If you need to reprocess historical data, replay the event log through the same streaming system.

Kappa Architecture trades the complexity of dual systems for the challenge of making stream processing robust enough to handle all use cases. With modern streaming engines and event logs, this is increasingly practical.

In this chapter, you will learn:

The Core Idea