A standard LLM call takes text as input and generates text as output. On its own, the model can't look up an order, run a test, or check current inventory. Tool use makes these actions possible. The model requests a tool call, your application executes it, and the result is sent back to the model to decide what to do next.

When the model can repeat this process and decide which steps to take, it becomes the foundation of an AI agent. This flexibility makes agents useful for tasks that require multiple steps or can't be fully planned in advance. But it also introduces new challenges: unpredictable costs, infinite loops, and errors that compound across multiple steps.

In this chapter, we'll explore how tool calling works, when to use fixed workflows instead of autonomous agents, and common patterns for organizing LLM calls. We'll also cover how to design reliable tools, use MCP to connect agents to external systems, coordinate multiple agents, and build safeguards that control costs, limit execution, and handle failures.

A tool-using agent follows a simple loop. You send the model a prompt along with definitions of the available tools. The model either responds directly or returns a structured request to call one or more tools. Your application executes the requested tools, adds their results to the conversation, and calls the model again. This continues until the model produces a final answer or the system reaches an execution limit.

In this example, the model makes two tool calls before it has enough information to answer. The important distinction is that the model never accesses the payment system directly. It only specifies which tool to call and what arguments to pass. The orchestrator validates the request, checks permissions, and executes the tool.

A simplified implementation looks like this:

Two details of this loop affect cost and security.

First, the conversation history is sent back to the model on every iteration. As the agent makes more tool calls, the input grows. A 10-step task may consume significantly more than 10 times the tokens of a single model call, especially when tools return large amounts of data. This makes context management an important part of controlling agent costs.

Second, execute_tool receives the current user's identity. The orchestrator must enforce that user's permissions for every tool call, rather than allowing the model unrestricted access through the application's service credentials.

Where the Loop Can Go Wrong

Each stage of the loop introduces potential failures:

Building a reliable agent also means handling these failures. You need ways to validate tool calls, control execution, manage context, recover from errors, and prevent the agent from running indefinitely.

We'll cover these mechanisms throughout the chapter. But first, there's a more fundamental design decision: does the task actually need an agent?

Systems that combine LLMs with tools generally fall into two categories: workflows and autonomous agents.

A workflow follows a predefined process controlled by your application code. The model handles individual tasks, such as classifying a support ticket or drafting a response, but your code determines which step runs next. The workflow can include branches and conditional logic, but the possible paths are defined in advance.

An autonomous agent has more control over how a task gets done. You give it a goal and a set of tools, and the model decides which actions to take, which tools to call, and when it has enough information to finish.

In a workflow, the execution path is controlled by code, making the number and order of model calls relatively predictable. In an agent, the model chooses its next action based on previous results. A task might take two steps or twenty, depending on what the agent discovers along the way.

This distinction affects several important design decisions:

A useful rule is to use the least amount of autonomy needed to solve the problem.

If you can define the steps for most requests, start with a workflow. For example, a refund request that always requires checking the payment, verifying refund eligibility, and processing the refund doesn't need an agent to decide what to do next. A predefined workflow is simpler, faster, and easier to test.

Agents become useful when the next step depends on what the system discovers during execution. Consider an agent fixing a failing test. It needs to inspect the error, locate the relevant code, identify the problem, make a change, and rerun the tests. If the test still fails, it must investigate further. The exact sequence of actions and number of iterations cannot be determined in advance.

Some production systems combine both approaches. A customer support system might handle common requests through predefined workflows and route more complex cases to an agent with access to additional tools. This keeps routine operations predictable while allowing greater flexibility where it's actually needed.

Between a single model call and a fully autonomous agent, several patterns can help you build more capable LLM systems without giving the model complete control. Most are structured workflows, and they're often sufficient for production use cases.

Prompt Chaining

Prompt chaining breaks a task into a sequence of steps, where each model call uses the output of the previous one. You can also add validation between steps to catch errors before they propagate.

For example, a system that generates product descriptions might first create an outline, verify that it includes all required features, and then generate the full description. Each call focuses on a smaller, well-defined task, which can improve output quality at the cost of additional latency.

Routing

Routing classifies an incoming request and directs it to the appropriate handler. For example, a customer support assistant might send billing questions to one prompt and toolset, technical issues to another, and refund requests to a workflow with stricter validation.

Routing can also reduce costs. Simple requests can use a smaller, faster model, while complex requests go to a more capable model. The routing decision can come from a small LLM, a trained classifier, or even simple rules when the categories are easy to distinguish.

Parallelization

Parallelization runs multiple model calls concurrently and combines their results. There are two common approaches:

Because the calls run concurrently, total latency is roughly determined by the slowest call rather than the sum of all calls, plus some coordination overhead. This lets you perform more analysis without increasing latency as much as sequential execution would.

Orchestrator-Workers

In the orchestrator-workers pattern, a model breaks a task into subtasks, assigns them to workers, and combines their results. Unlike sectioning, the subtasks aren't predefined. The orchestrator determines what work is needed based on the input.

In this example, the orchestrator analyzes the requested API change, identifies three components that need updates, and assigns a worker to each. Another request might require only two workers or as many as ten.

The workers can often run in parallel, but their changes still need to be integrated and validated before the task is complete.

Evaluator-Optimizer

The evaluator-optimizer pattern uses one model to generate an output and another to evaluate it against defined criteria. If the output doesn't meet those criteria, the evaluator provides feedback, and the generator tries again.

This works best when you can clearly define what a successful result looks like. For example, a code generation system can use an evaluator to review the implementation and an automated test suite to verify correctness. If the tests fail, the generator receives the errors and attempts a fix.

The important safeguard is to limit the number of iterations. Without a maximum number of attempts, the system can keep retrying a task it cannot solve. For many applications, two or three revision rounds are a reasonable starting point, though the right limit depends on cost, latency, and task complexity.

Choosing the Right Pattern

Here's how these patterns compare:

These patterns aren't mutually exclusive. A coding agent might first route a request, use an orchestrator to divide the work, run workers in parallel, and have each worker refine its implementation based on test results.

The right choice depends on how much flexibility the task requires and how much control you need over execution, latency, and cost.

Regardless of the pattern, the system's reliability depends heavily on the tools available to the model and how those tools are designed. That's what we'll look at next.

The model understands a tool through its name, description, and parameter schema. If the description is vague, the model may choose the wrong tool. If the schema is too permissive, it may generate incorrect arguments.

Schemas

A well-designed tool definition explains what the tool does, when to use it, and what arguments it accepts.

Several details help prevent common mistakes. The enum restricts status to supported values, so the model can't pass something like "in transit" when schema constraints are enforced. The parameter descriptions clarify the expected format and meaning of each argument. The tool description also explains when to use it and what results to expect.

For sensitive values like customer_id, the safer approach is to have the orchestrator inject them from the authenticated session rather than relying on the model to supply them.

Many LLM providers also support strict tool calling, which constrains generated arguments to a supported JSON schema. This helps prevent malformed JSON, missing required fields, and invalid enum values. However, it doesn't guarantee that an order ID exists or that the user has permission to access it. Your application must still validate arguments and enforce authorization.

Granularity

Tool granularity is a trade-off. Tools that are too narrow force the model to make several calls for a simple task, increasing latency and the chance of errors. Tools that are too broad become harder to describe, harder to use correctly, and potentially unsafe.

Design tools around meaningful tasks rather than individual internal API operations.

For example, if checking an order's delivery status requires calls to three backend services, expose a single get_delivery_status(order_id) tool rather than making the model coordinate those calls.

The last example also has security implications. An unrestricted SQL tool could allow the model to execute expensive queries, access sensitive data, or modify records. A narrowly scoped reporting tool limits the operations available to the model.

The number of tools also matters. Every tool definition consumes context tokens, and choosing the right tool becomes harder when dozens of tools have similar names or overlapping responsibilities.

For agents with large tool catalogs, consider exposing only the tools relevant to the current task. You can select them through routing or let the model search a tool registry before making a call.

Return Values

Tool responses become part of the model's context, so keep them focused.

Return the information needed for the next decision rather than passing through an entire API response with dozens of irrelevant fields. Large responses should be filtered, summarized, or paginated.

For example, instead of returning hundreds of orders, return the first 10 with a message:

This helps control token usage and prevents large tool outputs, such as logs or search results, from consuming the context window.

Tool responses should also be treated as untrusted input. External documents, search results, and API responses may contain instructions that the model should not follow. Validate and sanitize results where appropriate, and never treat tool output as a source of system-level instructions.

Error Messages the Model Can Act On

When a tool fails, the model needs enough information to understand the problem and decide what to do next.

A generic error such as:

doesn't explain what went wrong. The model may retry with the same arguments or give up entirely.

A more useful response is:

Now the model knows how to correct the request.

The same principle applies when a tool returns no results. Instead of returning an empty list, provide useful context when available:

This helps the model distinguish between a failed search and a customer who has no orders at all.

Good tool errors explain what went wrong and, when possible, how to recover. For production systems, structured error codes can make this even more reliable by distinguishing invalid arguments, missing resources, permission failures, and temporary service errors.

Avoid exposing stack traces, credentials, or internal implementation details in error responses. Tool outputs may be included in the model's final answer, so error messages should contain only information the model is authorized to see.

When multiple applications need access to the same external tools, integration work quickly becomes repetitive. Your chat assistant, coding agent, and internal support platform might all need access to the same ticketing system, with each team building and maintaining its own integration. With M applications and N tool providers, this can lead to M × N separate integrations.

The Model Context Protocol (MCP) addresses this problem by providing a standard way for AI applications to discover and use external tools. A provider can expose its capabilities through an MCP server, and any compatible application can connect to it without building a custom integration for that provider. Anthropic introduced MCP in late 2024, and several model providers and agent frameworks now support it.

MCP defines three main roles:

When an application connects to an MCP server, it can discover the tools that server provides, including their names, descriptions, and parameter schemas. The host makes the relevant tool definitions available to the model. When the model requests a tool call, the MCP client forwards it to the server and returns the result.

The model doesn't need to know whether a tool runs locally or calls a remote API. That complexity is handled by the MCP client and server.

Local vs. Remote Servers

MCP supports two main deployment approaches.

Local servers typically run as processes on the user's machine and communicate through standard input and output (stdio). They're useful for filesystem access, Git operations, and local development tools. However, they can access resources permitted to the process, so running an untrusted local server carries similar risks to installing untrusted software.

Remote servers run as network services and communicate using Streamable HTTP. A SaaS provider can host a shared MCP server for its customers, manage authentication centrally, and deploy updates without requiring users to reinstall anything.

Authentication and Authorization

For protected remote MCP servers, the specification defines an authorization framework based on OAuth 2.1. The user authorizes access through an authorization server, and the MCP client receives an access token intended for the MCP server. The server validates that token and enforces the user's permissions before executing tool calls.

This allows an agent to access external services on a user's behalf without giving it unrestricted access. For example, a support agent should only be able to retrieve tickets that the authenticated user is authorized to view.

MCP servers that use OAuth have two security requirements.

First, the MCP server must validate the token's intended audience. A token issued for another service must not be accepted, even if it's otherwise valid.

Second, the MCP server must not forward the client's access token directly to downstream APIs. If it needs to call another service, it should obtain separate credentials authorized for that service. This preserves authorization boundaries and prevents tokens from being reused across services they weren't intended for.

MCP standardizes how applications connect to tools, but it doesn't automatically make those tools safe. You still need permission checks, input validation, execution limits, and careful handling of tool results. A malicious or compromised MCP server can return content designed to manipulate the model, so external tool outputs must always be treated as untrusted data.

Some tasks are easier to handle with multiple specialized agents rather than one agent doing everything. In a multi-agent system, a lead agent breaks a task into smaller parts and delegates them to sub-agents, each with its own context window, instructions, and potentially different tools.

Each sub-agent can search multiple sources, process documents, and make several tool calls. Instead of passing everything back, it returns a focused summary of its findings. The lead agent combines these results without having to process every intermediate step.

This approach offers two main benefits.

Parallel execution. Independent subtasks can run concurrently. For example, a research task requiring ten separate searches can finish much faster when multiple sub-agents handle different searches simultaneously.

Context isolation. A single agent working on a long task can accumulate large amounts of information from tool calls, documents, and intermediate results. As the context grows, relevant information becomes harder to identify, and response quality can suffer. Sub-agents keep their intermediate work in separate contexts and return only the relevant findings, allowing the lead agent to stay focused on the overall task.

However, these benefits come at a cost. Anthropic reported that multi-agent research systems can consume roughly 15 times as many tokens as a standard chat interaction. Coordination also introduces new failure modes. Sub-agents might duplicate work, misunderstand their assignments, or produce summaries that omit important details. Debugging becomes harder because an incorrect final answer might originate from a mistake made several delegation steps earlier.

A single agent is usually preferable when tasks are small, highly sequential, or require shared context. Consider a coding task where changes to one file depend on decisions made in another. Independent agents may make conflicting changes unless their work is carefully coordinated.

Multi-agent systems work best for large, independent, read-heavy tasks, such as researching a topic across many sources or analyzing separate datasets. For tasks that require tightly coordinated changes, a single agent with effective context management is often simpler, cheaper, and more reliable.

An agent can decide which actions to take and how many steps to execute. Without limits, it might keep calling tools, repeat the same actions, or consume far more resources than expected. A production agent needs hard limits enforced by the orchestrator, because instructions in the prompt alone can't guarantee that it stops.

Step limits restrict the number of iterations an agent can execute. Once the limit is reached, the agent stops, returns its progress, or escalates to a human. A customer support agent might allow 10 steps, while a coding agent working on a complex task might need 100 or more.

Token budgets limit the total input and output tokens consumed during a run. Step limits alone aren't sufficient because each iteration can become more expensive as conversation history and tool results accumulate. Setting budgets per run, and potentially per user or tenant, helps prevent unexpected costs.

Timeouts should apply at two levels. Individual tool calls need timeouts so a slow or unresponsive service can't block execution indefinitely. The entire agent run also needs a deadline based on the application's latency requirements. An interactive assistant might have 30 seconds to respond, while a background research agent could run for 20 minutes.

Loop detection identifies agents that keep executing without making meaningful progress. A simple approach tracks recent tool calls and detects repeated calls with identical arguments. A more useful implementation also compares results. If an agent repeatedly searches for the same information and receives the same results, it may be stuck. The orchestrator can provide corrective feedback, change the execution strategy, or terminate the run.

Here's a simplified example:

This catches simple repetition, though repeated calls or identical results don't always mean the agent is stuck. Production systems may need more sophisticated progress checks, especially for tasks where retries are expected.

The appropriate limits depend on the type of agent and how it's used:

These values are illustrative starting points, not universal recommendations. In production, use execution traces to understand how many steps, tokens, and seconds successful tasks typically require. Set limits high enough to accommodate legitimate requests while stopping runs that consume resources without making progress.

Finally, how an agent handles a limit is just as important as the limit itself. Silently terminating a run leaves the user without a useful outcome. Instead, the system should explain what was completed, what remains unresolved, and whether the task can be continued or needs human intervention.

Agents can fail in ways that go beyond generating an incorrect response. Here are four failure modes to plan for and how to handle them.

Wrong Tool

The model selects a tool that doesn't fit the task or answers from its own knowledge when it should have called a tool. This often happens when tool descriptions overlap or the model has too many similar tools to choose from.

The best way to reduce these errors is through better tool design. Clearly describe when each tool should and shouldn't be used. Combine tools with overlapping functionality, and expose only those relevant to the current task. Track incorrect tool selections and add them to your evaluation set, since changes to prompts or models can introduce new errors.

Invented Arguments

The model generates arguments that were never provided, such as a plausible-looking order ID that doesn't exist or an email address guessed from a user's name.

Schema validation catches formatting errors but cannot detect invented values that look valid. For identifiers, verify that they came from a trusted source, such as an authenticated session or a previous tool result. Sensitive identifiers, like the current user's ID, should be injected by the orchestrator rather than generated by the model.

For tools that modify data or trigger external actions, validate the arguments, permissions, and current state before execution.

Infinite Loops

The agent repeatedly calls the same tool or cycles between several actions without making progress. This often happens when a tool returns an unclear error, the agent keeps retrying a failed operation, or the task cannot be completed with the available tools.

Step limits and loop detection prevent these situations from continuing indefinitely. Clear, actionable error messages help the agent recover instead of repeating the same mistake. The system prompt should also make it clear that the agent can stop and explain why a task couldn't be completed rather than continuing to retry.

Compounding Errors Over Long Tasks

Compounding errors are hard to catch because individual steps may look correct even when the overall task goes wrong.

Suppose an agent completes each step correctly with 95% probability. Assuming independent outcomes, the probability of completing all 20 steps correctly is:

\[

0.95^{20} \approx 36\%

\]

This is a simplified example, but it illustrates how reliability can decline as tasks get longer. An early mistake, such as misunderstanding a requirement, can affect every subsequent step that depends on it.

Several techniques help reduce this risk:

These techniques won't eliminate every failure, but they make errors easier to detect and recover from.

When a system design problem involves tool use, start by deciding how much autonomy the system actually needs. If most requests follow predictable steps, propose a workflow and explain why it's sufficient. If the next steps depend on intermediate results, consider an agent and clearly define which decisions it can make and which remain under application control.

Next, explain which tools the system needs, which can modify data or trigger external actions, and how permissions are enforced for every tool call. Cover execution limits, loop detection, error handling, and how the system verifies results before proceeding.

Use concrete numbers to make your design decisions clear. For example, "Each run is limited to 15 steps and 100K tokens, after which unresolved requests are escalated to a human" is more useful than simply saying you'll add safeguards. Treat these numbers as initial assumptions and explain how you'd refine them using production metrics.

Agents allow LLMs to interact with external systems and carry out multi-step tasks. But every tool call adds cost, latency, and potential failure points. Here are the key design considerations:

Start with the simplest design that meets the requirements, give the model well-defined tools, and enforce step, token, and time limits in code so the agent stops safely when it can't complete a task.