LLMs generate text. They read the context you send them and predict the next tokens. That is useful on its own, but many applications need the model to do more than write an answer.

Real AI applications often need current data or access to systems outside the model: search indexes, calendars, ticketing systems, customer records, code execution, payment systems, or internal APIs. The model should not directly operate those systems. It should request an action in a structured way, and your application should decide whether that action is allowed and how to run it.

That separation is the heart of tool use. The model proposes. Your code validates, authorizes, executes, logs, retries, and handles errors. This keeps the model useful without giving it unchecked access to real systems.

In this chapter, you will build the basic loop behind function calling: define tools, let the model request one, execute it in your code, return the result, and ask the model to write the final answer.

In the previous module, you saw how RAG gives a model access to external knowledge by retrieving documents at query time. Retrieval helps with stale or private knowledge, but it is mostly read-only. It does not create a calendar event, update a CRM record, issue a refund, or send a Slack message.

Consider what a plain model call can and cannot do by itself:

The pattern is simple: if the task depends on live state, private data, or a real side effect, text generation is not enough. The model can describe the action. Your application has to perform it.

Function calling lets the model return a structured tool request instead of only natural language. A tool request usually contains a function name and arguments. Your application parses that request, validates it, executes the mapped implementation, and sends the result back to the model.

For custom tools, the model requests execution; it does not execute the tool itself. Your code still owns authentication, authorization, validation, rate limits, logging, retries, side effects, and error handling. Some providers also offer hosted tools such as web search, file search, code execution, computer use, or MCP connectors. Even then, there is still a controlled execution layer between the model's request and the outside world.

The diagram shows the two common paths. If no external state is needed, the model can answer directly. If the request needs live data or a side effect, the model emits a tool call. Your code executes the call, returns the result, and the model uses that result to answer.

Function calling has a straightforward control flow. Provider APIs differ in naming and wire format, but the engineering pattern is the same.

Phase 1: You define the available tools

When you send a request to the model, you include tool definitions alongside the user's message. Each definition describes one capability: its name, what it does, and the shape of its arguments.

Phase 2: The model decides whether to call a tool

Based on the user request and the available tools, the model either responds directly or emits one or more tool calls. In the Chat Completions API this appears as tool_calls; in the Responses API, function calls appear as output items. In both cases, the important pieces are the tool name and the arguments.

Phase 3: Your code executes the function

Your application receives the tool call, parses the arguments, validates them against your own rules, and runs the mapped implementation. This might call a weather API, query a database, enqueue a job, or create a record in an external system.

Phase 4: You send the result back

You send the tool result back to the model using the provider's expected format. In Chat Completions, that is usually a tool message with the matching tool_call_id. In the Responses API, it is a function-call output item tied to the original call. The model then uses the data to produce the final response or request another tool.

Here is a sequence diagram showing this full loop:

This loop adds an extra step. In a client-managed loop, the first request gets a tool call and the second request sends the tool result back so the model can answer. Some APIs can manage hosted tools inside a single response request, but custom function execution still creates an application boundary. That boundary is where you handle latency, retries, permissions, and logging.

Tool definitions are documentation for the model. Vague schemas lead to vague behavior: wrong tool selection, missing arguments, unsupported enum values, and avoidable clarification questions. Write tool descriptions with the same care you would give an API used by another engineer.

Each tool definition has three parts:

Here is what a complete tool definition looks like in the Chat Completions tool format:

The name should clearly describe the action: get_current_weather, search_users, create_ticket. Verb-noun names are a good default because they map cleanly to user intent.

The description is the most important field. It should say what the tool does, what it returns, when to use it, and how it differs from similar tools. Prefer plain engineering language over vague product language. "Get current weather conditions for a specific location" is useful. "Provides powerful weather intelligence" is not.

The parameters use JSON Schema. Each property should have a type (string, number, boolean, array, object) and, for important fields, a short description. The enum field restricts values to a specific set, which reduces unsupported options. The required array lists which parameters must be provided.

Here is a more complete example with multiple related tools:

The three tools cover different time horizons: current, future, and past. The descriptions make that distinction explicit. A user asking "What will the weather be like in Paris next week?" needs the forecast tool, not the current-weather tool. The model is much more likely to choose correctly when the descriptions say exactly when each tool applies.

When tool selection fails, inspect the schema before blaming the model. Add clearer usage guidance, negative cases, tighter parameter constraints, and test cases for the requests that caused confusion.

Now put the pieces into a runnable loop. The core pattern is: send the conversation and tool definitions, inspect the response for tool calls, execute any requested tools, append their results, and repeat until the model returns a final message.

Here is the complete implementation:

Here is the execution path for the first question:

The while loop matters. The model may call several independent tools in one round, or it may ask for another tool after seeing the first result. The loop continues until the response contains no more tool calls.

The tool_call_id field matters when sending results back. Each tool call has a unique ID, and the result must reference that ID so the model knows which call produced which result. If you are handling multiple parallel tool calls, getting the IDs wrong will confuse the model.

When you send tool definitions, the model uses them as part of its context. It compares the user's request with the available tool names, descriptions, and parameter schemas, then chooses whether a tool call is more appropriate than a text response.

This is not a keyword match. "Is it going to rain tomorrow in Berlin?" should map to get_weather_forecast even though the user never said "forecast." But the model is still sensitive to the schemas you provide. Similar names, overlapping descriptions, or missing negative cases increase the chance of a wrong call.

The tool_choice Parameter

You can influence tool selection using tool_choice. Exact values vary by provider and API mode, but the common controls are:

Here is how to use each option:

When do you need to override tool_choice?

Most applications start with "auto". Override it when your application already knows the right mode:

When the Model Gets Confused

Tool selection can still fail in predictable ways:

The first fix is usually schema work. Add negative cases ("Do not use this for forecast questions"), explicit triggers ("Use only for current conditions"), tighter enums, and tests that represent the ambiguous cases your users actually ask.

Here is a complete interactive weather assistant. It keeps conversation history, supports multi-turn follow-ups, executes requested tools, and returns structured errors instead of crashing on unknown functions.

Sample Output:

Try this sequence to see how context affects tool choice:

The fourth message matters because it should not call a weather tool. When no tool fits the request, the model should answer from its training data instead of forcing a call. Knowing when to stay in plain text is part of using tools well.

References