Repartitioning and Internal Topics

An order service publishes events keyed by order ID. That key keeps updates for an order together, but a reporting application wants to count orders by store. The producer can send orders from the same store to several input partitions.

Changing the key in application code gives each record the right label. The records must also reach the processing task responsible for that store's count. Kafka Streams uses repartitioning to make that happen.

This chapter explains when a topology needs repartitioning, how internal topics support it, and how those topics differ from the changelogs Kafka Streams uses to recover state. We'll follow a small order-counting topology and examine its effects on serialization, parallelism, and operation.