Broker Failure and Recovery

A broker can fail while producers are sending records, consumers are fetching data, and followers are copying partition logs. Kafka may restore application traffic before that broker returns. Even after the process restarts, its replicas may need time to recover and catch up.

Understanding those stages helps you distinguish restored service from restored redundancy. We’ll follow a broker outage through detection, client impact, restart, and verification, then examine what changes when the broker loses its storage.

The examples use Apache Kafka 4.3 with KRaft, three brokers, and a separate, available controller quorum. Assume local storage, non-transactional records, and consumers reading from leaders. Partition recovery uses eligible replicas that preserve committed history; promoting a stale replica through unclean election can have different, data-losing outcomes.