Event Streaming with Kafka

Traditional queues deliver messages and then remove them. That works when each message represents work to be done once, but it makes replay and multiple independent consumers harder.

Kafka uses a different model: an append-only log. Producers add events, and consumers read at their own pace while tracking their own position. Reading does not delete the event, so it can be replayed or consumed by many services.

This chapter explains Kafka’s log model and why replay and throughput are the main reasons to choose it over a queue.