Latency in AI applications is a pipeline property. The LLM call matters, but so do embedding generation, vector search, document fetches, prompt assembly, tool calls, validation, retries, and network placement.

Teams often optimize the model call first because it is the easiest part to see. That helps when the model is the bottleneck. In RAG and agent systems, though, the slow request is often a chain of smaller delays: one service call after another, a slow document fetch, a cold cache, or a retry that was never logged.

Latency optimization does not always require new infrastructure. Streaming makes the application feel faster. Parallel execution removes unnecessary waiting. Caching and pre-computation avoid repeated work. Prompt changes and model routing can reduce both cost and wait time when you validate them against real tasks.

The right approach is measurement first, then targeted changes. Optimizing without a latency breakdown usually moves the wrong number.