Distributed tracing records the path of a request or workflow through instrumented parts of a system.

In distributed systems, metrics can show that latency increased and logs can explain what happened inside individual services, but neither provides the full request timeline by itself. Traces show each significant operation, how long it took, which operation called which dependency, and where errors were recorded.

In this chapter, you will learn how traces, spans, context propagation, instrumentation, sampling, and tracing backends reveal request timelines across distributed systems.

A distributed trace is a record of one request or workflow as it moves through a distributed system. It is made of spans. Each span represents one operation, such as handling an HTTP request, calling another service, executing a database query, publishing a message, or processing a queue item.

Traces are only as complete as your instrumentation and sampling policy allow. They do not observe uninstrumented code, and in production you usually keep only a subset of all traces. That is fine. You do not need every trace to debug most problems; you need enough representative traces, plus the important outliers.

From this trace, you can see that the request took 850ms end to end, the order service consumed most of that time, the database write was the slowest child operation, and auth, inventory, and notification work were not the primary cause.

Without tracing, you would piece this together from logs, timestamps, and guesses about which service called which dependency. With tracing, the timeline is explicit.