Committing and Managing Offsets

A consumer writes an order to a database, then crashes. On restart, should it read that order again or move to the next one? Kafka answers using the group’s saved offset, which may describe a different point in the stream from the application’s completed work.

Offset management connects processing to recovery. We’ll examine automatic and manual commits, partial failures, asynchronous commit results, and deliberate changes to reading positions. The aim is to make each saved position mean exactly what the application intends.

The examples use Java 21 and the Apache Kafka Java client and tools version 4.3.1. We’ll use ordinary consumer groups, non-transactional order records, and Kafka-based offset storage. Examples involving application threads assume one owner thread makes consumer API calls.