Tool Calling

The round trip

1. The application sends the conversation plus tool definitions: a name, a description and a JSON schema for the arguments, for example get_weather(city: string, date: string).
2. The model, instead of answering in prose, emits a call: {"name": "get_weather", "arguments": {"city": "Kathmandu", "date": "2026-10-06"}}. This is structured output, often marked by special tokens in the chat template.
3. The application validates the arguments, checks the user is allowed to do this, and runs the function.
4. The result is appended as a tool message: {"temp_c": 22, "rain_chance": 0.6}.
5. The model generates again, now able to read the result: "Expect about 22 °C with a 60% chance of rain."

The model never executes anything. In the notes under his introduction-to-LLMs video, Karpathy describes the mechanism: the model emits special words, the program running it detects them, sends the request to the tool, and continues generation with the result; fine-tuning data teaches when and how to call tools Established.

How models learn it

Toolformer taught a model to call a calculator, a question-answering system, search engines, a translation system and a calendar in a self-supervised way, keeping only the calls whose results helped predict the following text Established. Commercial APIs followed: OpenAI announced function calling on 13 June 2023, letting developers describe functions so the model can choose to output a JSON object containing arguments to call them Established. Today's models are post-trained on many examples of tool definitions, calls and results.

Retrieval is a tool too

A RAG pipeline that searches on every question is fixed. Expose search as a tool instead and the model decides when to search and what query to send, and can search again if the results are poor. That's a step towards agents.

Effects and risks

Once a model can call send_email or delete_file, its mistakes have consequences. And the results it reads can be hostile: OpenAI's announcement itself noted a proof-of-concept exploit in which untrusted data from a tool's output instructs the model to perform unintended actions Established. The defences are ordinary engineering: least privilege per tool, validation, confirmation before irreversible actions, and treating tool output as data (guardrails).

One call, then a loop

Everything here is one round trip. Let the model call tools repeatedly, reading each result and deciding what to do next until the task is done, and you have an agent, the subject of Chapter 13. The Model Context Protocol, which Anthropic open-sourced in November 2024 Established, standardises how tools and data sources are offered to models, so one integration works across applications.