Producer Retries and Failure Handling

A producer can lose its connection while publishing an order event. Sometimes Kafka has not received the event. Sometimes Kafka has stored it, but the response never reaches the producer. Both situations can initially look like the same failure to the application.

Retries help a producer recover from temporary problems, but they do not remove uncertainty or decide what the business should do when publishing ultimately fails. A reliable publishing path needs both client recovery and an application policy for unfinished work.

In this chapter, we’ll examine retries in the Apache Kafka Java client 4.3.1, explain the relevant time limits, and build a small helper for observing send results. We’ll use ordinary, non-transactional order events with acks=all and idempotence enabled unless stated otherwise.