Exercise: Agentic RAG
Exercise 1: Route a Query Before Retrieving
Build a router that decides how to handle each incoming query before doing any retrieval: answer directly when the question is general knowledge or chit-chat, and retrieve from the knowledge base only when the question is about the specific documents. Then act on that decision.
Standard RAG retrieves for every query, which wastes a retrieval call on "hello" and can pollute the context with irrelevant chunks for questions the model could answer on its own. Agentic RAG starts by letting the model classify the query and choose an action. This routing step is the simplest form of agency in a RAG system, and it is what keeps retrieval focused on the questions that actually need it.
Requirements
- Ask the model to classify the query as
retrieve(needs the knowledge base) ordirect(answer without retrieval). - If the decision is
retrieve, select the most similar chunk from the corpus and answer from it. - If the decision is
direct, answer without using the corpus. - Print the decision and the answer for each query.
Expected Output
The knowledge-base question is routed to retrieval; the thank-you message is answered directly.
Solution
The router is a small classification call that runs before any retrieval, turning "always retrieve" into "retrieve when it helps." That decision changes the control flow: a retrieve query embeds and searches the corpus, while a direct query skips straight to generation. Extending this idea (more actions, multiple retrieval sources, a loop that re-routes after seeing results) is what grows a RAG pipeline into an agent.
Exercise 2: A Three-Way Router with Clarification
A two-way retrieve-or-answer router still struggles with vague input. Add a third route: when a query is too ambiguous to act on, the agent asks for clarification instead of guessing. This is the difference between an agent that confidently answers the wrong question and one that knows when to ask.
Requirements
- Classify each query as
retrieve,direct, orclarify. - For
retrieve, fetch from the corpus and answer; fordirect, answer without the corpus; forclarify, return a clarifying question. - Print the route and the response for each query.
Expected Output
The factual query is retrieved, the thank-you is answered directly, and the vague "help me with it" triggers a clarifying question.
Solution
The third route encodes a basic competence: recognizing when there is not enough information to act. A vague query routed to retrieve would fetch an arbitrary chunk and answer confidently about the wrong thing; routing it to clarify keeps the agent honest. Adding routes this way is how an agentic system grows, each new action handles a case the simpler router got wrong, and the router stays a cheap classification call in front of the expensive work.