Heartbeats and Consumer Liveness

When a consumer stops responding, Kafka needs to let another member take over its partitions. Waiting too long leaves work unattended. Reacting too quickly can remove a consumer that paused only briefly and create unnecessary assignment changes.

Heartbeats help Kafka make that decision, but they answer only part of the problem. A consumer can keep its network activity alive while its application gets stuck processing records. Kafka therefore checks both membership communication and the application’s return to its poll loop.

In this chapter, we’ll follow those checks through a crash and a slow-processing incident. We’ll use a KRaft-based Kafka 4.3 cluster and the Apache Kafka Java consumer with automatic assignment through subscribe(). Our order-reporting group reads orders.placed. Unless stated otherwise, its consumers use dynamic membership, with no group.instance.id configured.