Partitioners and Record Routing

An order service publishes an order’s creation, payment, and cancellation events. If those events need the same partition so consumers can process them in order, the producer must route them consistently. At the same time, it needs to spread different orders across partitions so one partition does not carry all the work.

Partitioning connects these two concerns: which records belong together, and how producers distribute publishing work. A routing choice affects the log a record enters, the consumers that can process it, and what happens when the topic grows.

In this chapter, we’ll examine how the Apache Kafka Java client 4.3.1 chooses partitions. We’ll use an orders.events topic to compare keyed, unkeyed, explicit, and custom routing, then follow what changes during broker failure and partition expansion. Other clients may offer different algorithms or defaults, so the Java client’s behavior is not a universal Kafka protocol rule.