Reassigning Partitions and Scaling Clusters

Adding a broker gives a Kafka cluster more resources, but existing data does not automatically move onto it. The new broker may remain lightly loaded while the original brokers continue serving the same partitions.

Replica reassignment is how an operator changes that placement. Kafka copies data to the destination while the cluster continues serving traffic, then removes replicas that no longer belong in the assignment. The migration itself consumes capacity, so the route to a better layout matters as much as the final layout.

In this chapter, we’ll plan and execute a small reassignment, control its load, and verify the result. We’ll also examine leader placement, cancellation, and the additional work required to remove a broker safely.