Sequential I/O and Kafka Performance

A broker can store a large stream of order events without performing a separate random disk update for every order. Kafka appends record batches to partition logs, and consumers usually read forward through those logs. Those access patterns help Kafka move large amounts of data with relatively little work per record.

Sequential I/O explains an important part of Kafka's performance, but it does not make storage costs disappear. We'll look at why the pattern helps, how batching strengthens it, and what happens when replication, replay, and many partitions compete for the same resources.

Our examples use a KRaft-based cluster with local storage, tiered storage disabled, and regular consumer groups reading from partition leaders. The orders.placed topic contains non-transactional records. All numerical examples are illustrative calculations, not measured Kafka benchmarks.