Consumer Poll Loop

A consumer can run well for months and then lose its partitions when a database becomes slow. The broker is reachable, the process is alive, and the application is still processing records. The problem is that the application takes too long to return to its next poll.

The poll loop connects Kafka’s delivery of records to the application’s processing pace. We’ll examine what each iteration does, how its timing works, and how to handle failures without skipping unfinished records.

The examples use Java 21 and the Apache Kafka Java client 4.3.1, with an ordinary subscription-based consumer group. We’ll assume group.protocol=classic, automatic commits disabled, and non-transactional order events. The main example processes records sequentially on the consumer’s owner thread.