Offset and Timestamp Indexes

A reporting consumer needs to resume at offset 1032. Another application wants to replay orders starting at 09:00:07. Both requests describe where to read, but neither supplies a position in a Kafka data file.

Kafka uses offset and timestamp indexes to narrow the search, then examines the log data to find the relevant records. We'll follow both requests through one segment and see why the indexes stay small, how they handle batches, and what a timestamp lookup actually returns.

Our examples use a KRaft-based cluster, local storage without tiered storage, and the modern record batch format. The records in orders.placed partition 0 are non-transactional, committed, and still retained. We use producer-supplied Kafka timestamps through message.timestamp.type=CreateTime.