Kafka Metrics and Observability

An order-tracking dashboard is several minutes behind. Kafka’s brokers are running, producers report successful writes, and consumers are receiving records. Those signals are useful, but none explains why customers cannot see their latest order status.

Observability means using evidence from a running system to understand its behavior. For Kafka applications, that requires visibility into brokers, controllers, clients, and the work the application performs after a record arrives.

In this chapter, we’ll choose useful signals, collect them with clear meanings, and connect metrics, logs, and traces during an investigation. We’ll also look at dashboards and alerts that help an operator decide what to do next.