Metrics can tell you that latency went up. Logs can tell you what happened inside a single service. But when one request travels through ten services before returning a response, neither clearly shows where the time actually went. Distributed tracing fills that blind spot.

A trace records the full journey of a request as it moves through the system. It captures each important operation, how long that operation took, which service called which dependency, and where errors happened. Then it stitches those pieces into one timeline. Instead of guessing which step was slow, you can see it.

This chapter covers how traces, spans, trace context, instrumentation, sampling, and tracing tools work together to rebuild request timelines across a distributed system.

A distributed trace is a record of one request or workflow as it moves through a distributed system. It is made of spans. A span is one timed operation, such as handling an HTTP request, calling another service, running a database query, publishing a message, or processing a queue item.

Traces are only as complete as the code you instrument and the traces you keep. They cannot show code that does not create spans, and in production you usually keep only some traces. You do not need every trace to debug most problems. You need enough normal examples, plus the important unusual ones: errors, slow requests, and critical user flows.

From this trace, you can see that the request took 850ms end to end, the order service used most of that time, the database write was the slowest child operation, and auth, inventory, and notification work were not the main cause.

Without tracing, you would piece this together from logs, timestamps, and guesses about which service called which dependency. With tracing, the timeline is visible.