A language model is often first experienced through chat: ask a question, get an answer, move on. That interface is useful, but many real systems need more than one response. They need to inspect files, query APIs, compare options, recover from errors, or keep working until a task is done.

An AI agent is a software system that uses a language model to help choose actions toward a goal. The model may plan, inspect state, call tools, read tool results, revise its approach, and decide when to stop. The tools are ordinary software interfaces: APIs, databases, search systems, file systems, browsers, code execution sandboxes, or internal services.

The important distinction is control flow. In a chatbot, the application decides when the model is called and what happens next. In an agent, the model participates in choosing the next step. That extra autonomy can be useful, but it also creates new failure modes. This chapter explains the agent loop, the autonomy spectrum, and the engineering trade-offs that determine when an agent is the right design.

A chatbot's flow is simple. The user sends a message, the application calls the model, and the model returns a response. That is one round trip. Unless the application adds tools or memory, the model does not inspect external state, verify its answer, or recover from a failed action.

An agent runs a feedback loop. It observes the current state (what tools returned, what errors occurred, what the user asked), decides what to do next (which tool to call, what to search for, whether the goal is done), and acts (calls a tool, asks a question, returns an answer, or stops). Then it observes the new state. This continues until the agent reaches the goal or hits a limit.

The diagram captures the key difference: the agent loop has feedback. After each action, the system records what happened and uses that observation to choose the next step. The agent is not only producing text; it is operating inside a stateful control loop.

This loop is what makes agents useful for tasks where the path is not known ahead of time. An agent asked to compare flight options could search flights, notice that the first results are expensive, try nearby dates, check baggage fees, and summarize the trade-offs. A plain chatbot can discuss that strategy, but it cannot execute and adapt unless the surrounding application gives it those capabilities.

The same loop creates risk. An agent can call the wrong tool, repeat a search that is not helping, burn through a budget, or take an irreversible action from a mistaken premise. Every iteration costs tokens, latency, and operational attention. Every tool call is another place where validation, permissions, and error handling matter.

Not all AI systems are equally autonomous. It helps to think of them on a spectrum, from chatbots with application-controlled flow to agents that can pursue narrow goals with little human involvement.

Chatbot (application-controlled)

The model responds to one message at a time. It does not choose tools, maintain task state, or take external actions unless the application adds those capabilities. This is predictable, less expensive, and easier to reason about. Many customer support assistants and Q&A systems should stay here.

Copilot (human in the loop)

The model can suggest actions and even draft tool calls, but a human approves before anything important is executed. GitHub Copilot suggesting code edits is a copilot. An AI assistant that drafts a customer refund but waits for a support agent to click "approve" is a copilot. Copilots are a good default when the cost of a wrong action is high, such as modifying production data or sending messages to customers.

Supervised agent (human oversight)

The agent takes low-risk actions on its own, but a human can intervene or review important decisions. It might handle a sequence of read-only steps without asking, then pause at a decision point: "I found three conflicting records. Which one should I use?" Many production agents belong here: bounded autonomy, clear audit trails, and explicit escalation paths.

Autonomous agent

The agent pursues a narrow goal with little or no human involvement during the run. It decides what to do, handles expected errors, and reports results when done. This offers the most autonomy and the largest potential impact. It requires reliable tools, narrow task scopes, strong permissions, and robust failure handling. In practice, this level is rare outside narrow, well-bounded tasks.

Where you sit on this spectrum is primarily a risk decision. Ask: what is the cost of a wrong action, how quickly can it be detected, and how easily can it be reversed? A wrong answer in a chatbot may be embarrassing. A wrong action in an autonomous agent may delete records, send incorrect emails to customers, or place orders. Match autonomy to impact, reversibility, and your ability to audit the run after the fact.

A common mistake in AI engineering is using an agent when a workflow would be simpler and more reliable. Agents introduce variable latency, harder testing, and less predictable behavior. Before building one, ask whether the model truly needs to decide the next step.

Here is a decision framework:

Chatbots are the right default for conversational interfaces. If the task can be answered in one model call, do not add complexity.

Pipelines are sequences of steps where you, the developer, decide the order and branching logic in advance. You call the model at step 1, pass the result to a parser or service at step 2, feed the structured result to another model call at step 3, and so on. The control flow lives in your code, not in the model's judgment. Pipelines are easier to test, monitor, and control costs for. If you can define the steps upfront, use a pipeline.

Agents are the right choice when the task requires adaptive decision-making, where the next useful step depends on intermediate results in ways you cannot list cleanly at design time. A research agent that searches, reads, evaluates relevance, and changes strategy based on what it finds may need an agent loop. A fixed three-step summarization workflow does not.

A common mistake is building an agent for a task that is really a pipeline. The model does not need to decide the order, you already know it. Forcing the model to make decisions it does not need to make wastes tokens, adds latency, and introduces failure points. Build the simplest thing that works, and only add agent-level autonomy when the task genuinely requires it.

Once you have decided to build an agent, you need to choose an architecture. Three common patterns are single-loop agents, hierarchical agents, and collaborative agents. Each one trades complexity for capability.

Single-Loop Agent

This is the simplest architecture: one model, one loop, one set of tools. The model receives a goal, calls tools as needed, observes results, and keeps looping until it decides it is done. This is close to the ReAct pattern (reasoning and acting), and it is what many engineers mean by a basic agent.

Single-loop agents work well for tasks that fit in one context window, use a modest number of tool calls, and do not need separate specialist roles. A focused research assistant, a coding assistant working on one bug, or a support agent with read-only access to order APIs can often be built this way.

The limitation is state management. Every tool result competes for context. After enough iterations, the model may carry too much irrelevant history or miss important early constraints. Long-running tasks need summaries, external state, checkpoints, or a different architecture.

Hierarchical Agent

A hierarchical agent has an orchestrator that delegates to sub-agents. The orchestrator breaks the goal into subtasks, assigns each one to a specialized sub-agent, collects the results, and synthesizes them into a final answer.

For example, a competitive analysis agent might have:

Each sub-agent operates in its own context window. This helps with context limits and allows specialization. The research sub-agent can have a prompt and toolset built for web research. The synthesis sub-agent can have a prompt built for comparative analysis.

The trade-off is complexity. You now have multiple model calls, each adding cost and latency. The orchestrator's output quality directly limits the system's overall quality. If the orchestrator breaks down the task poorly, every downstream agent works from bad instructions.

Collaborative Agent

Collaborative agents (also called multi-agent systems) are a network of agents that communicate with each other, each owning a part of the overall task. Unlike hierarchical agents where control flows top-down, collaborative agents can pass work back and forth, check each other's outputs, or run in parallel.

This architecture can fit software development workflows, where a planning agent defines the task, a coding agent writes the implementation, a review agent checks it, and a testing agent runs tests. If the tests fail, the testing agent can pass the failure back to the coding agent.

Collaborative architectures are the hardest to build and debug. They are justified when the task has genuinely different modes of work, such as planning, implementation, review, and test execution, and when independent verification is worth the coordination overhead.

For most projects, start with a workflow or a single-loop agent. Add hierarchy or collaboration only after you can point to the specific limitation it solves.

Agents are not free. Before you commit to building one, have a clear view of the costs and the ways they fail.

Token cost

Every iteration of the agent loop sends context back to the model. A 10-step loop can use far more tokens than a single response because later calls include earlier messages and tool results. Avoid hard-coding pricing assumptions in your design; model prices, context windows, and cached-token discounts change. Benchmark real runs, compute cost from provider pricing at deployment time, and set per-run budgets before exposing an agent to high traffic.

Latency

Each model call adds network latency, and each tool call adds the time needed to execute the tool itself. Multi-step agent runs can be noticeably slower than a single chatbot response. For real-time user-facing interactions, that may be unacceptable. Agents often fit background tasks and asynchronous workflows better than instant responses.

Reliability and error propagation

In a pipeline, an error at step 3 usually fails at a named step with a clear stack trace. In an agent loop, the error becomes part of the model's next observation. The agent may recover, but it may also misread the error, try a dangerous workaround, or spend several iterations on an unproductive path. Agent reliability depends on defensive orchestration: validation, timeouts, budgets, retries, explicit stop conditions, and human escalation for high-impact actions.

Agents trade reliability and cost for capability. They can do more, but they cost more to run, fail in more ways, and are harder to debug when they do. Match the tool to the task.

Let's build one. The core of a simple agent is a loop that keeps calling the model until the model returns a final answer or the system stops it. An agent loop extends function calling: the model can request tools repeatedly, and the application feeds each tool result back into the next model call.

The key difference from a single tool call is the stopping condition. The loop continues as long as the model keeps requesting tools, up to a limit you control.

Let's walk through what this code does.

The run_agent function takes a goal and a maximum number of iterations. It initializes the message history with a system prompt and the user's goal, then enters the agent loop.

On each iteration, it calls the model and checks finish_reason. If the reason is "stop" (or there are no tool calls), the model has decided it has enough information and produced a final answer. The loop exits and returns that answer.

If the model produces tool calls, the code executes each one, collects the results, and appends them to the messages array. Then it loops back to call the model again with the updated context.

The max_iterations guard is important. Without it, a confused model could keep calling tools. When the limit is hit, the code makes one final model call asking for the best available answer instead of failing silently.

Run this with the sample goal and you will see the agent call search_web to retrieve example pricing, then call calculate to work out the cost for 500,000 tokens, and finally produce an answer that combines both tool results. In a real product, you would use live pricing from configuration or a provider billing API, not a hard-coded lesson value. You would also replace the small calculator example with a proper parser, a trusted math library, or a sandboxed execution environment.

References