Kafka Cluster Startup and Metadata Changes

A Kafka process can be running before it is ready to do useful work. Controllers need to establish metadata leadership. Brokers need to register, learn the cluster state, and recover their local logs. Applications need reachable leaders for the partitions they use.

The same distinction appears after startup. A topic creation request can succeed while some brokers and clients are still learning about the new topic. Understanding these stages helps explain delays that would otherwise look like inconsistent cluster behavior.

This chapter follows a KRaft cluster from startup into normal metadata updates. The examples use Kafka 4.3, dedicated Controllers 101, 102, and 103, and Brokers 1, 2, and 3.