Idempotent Producers

A producer sends an order event, Kafka stores it, and the acknowledgment disappears when the connection breaks. The producer has an uncertain result: retrying may repeat a successful write, while giving up may abandon a write that never happened.

An idempotent producer lets the Kafka client retry without appending another copy of the same send. Kafka recognizes the retry using producer identity and sequence information the records carry.

In this chapter, we’ll follow that mechanism, configure a Java producer, and examine the limits of its guarantee. The examples use Apache Kafka 4.3.1 and its Java client with a KRaft-based cluster. They cover non-transactional production; consumer processing and external side effects remain separate responsibilities.