Agents, tool use and orchestration

From answering to acting

The systems in the previous sections produce text. Many useful systems must also act: look up an order, search a knowledge base, issue a refund, book a table, run a test suite. Tool calling (often called function calling) is the mechanism. The application describes the tools available, each with a name, a description and a JSON Schema for its parameters. Instead of answering in prose, the model may return a structured request to call one or more of them with particular arguments. The application, not the model, decides whether to execute the call, executes it and returns the result to the model, which then continues. The model never touches a database or an API itself; every effect in the world passes through code you wrote.

An agent is a system in which the model chooses its actions repeatedly: it calls a tool, reads the result, decides what to do next, and continues until the goal is reached or a limit stops it. Autonomy is a spectrum. At one end, code calls the model once to fill in a form; at the other, a model plans and executes a long task with dozens of tool calls and little supervision. Architecturally the important point is that every step towards autonomy moves decisions from deterministic, reviewable code into a probabilistic component, which increases flexibility and decreases predictability, testability and control. That trade-off should be made deliberately, and as late and as little as the problem requires.

{
  "name": "get_order_status",
  "description": "Look up the current status of one order placed by the signed-in customer. Use when the customer asks where an order is. Returns status, estimated arrival and courier position. Changes nothing.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "pattern": "^ORD-[0-9]{6}$",
        "description": "Order number, for example ORD-104233"
      }
    },
    "required": ["order_id"],
    "additionalProperties": false
  }
}

The agent loop: reason, act, observe

The loop behind almost every agent was described by Shunyu Yao and colleagues in ReAct: Synergizing Reasoning and Acting in Language Models (2022). ReAct interleaves reasoning (the model writes down what it is trying to do), actions (it calls a tool) and observations (the tool's result is added to its context), so that each step is informed by what the previous one found. Before ReAct, models either reasoned without access to facts, and hallucinated, or acted without reasoning, and wandered. Today's tool-calling interfaces and reasoning models build this loop in, but its structure is still the one to design around.

In code the loop is short. Build the context (instructions, goal, history, tool definitions, results so far); call the model; if it returns a final answer, stop; if it returns tool calls, validate each against its schema and the business rules, check the user is entitled to it, execute it with a timeout, append the result (or a clear error message) to the context, check the limits and go round again. The diagram below shows where deterministic code surrounds the model. Everything outside the model box is ordinary software that can be tested, reviewed and enforced; that is where an agent's reliability and safety come from.

Workflows versus agents

In Building effective agents (December 2024), Erik Schluntz and Barry Zhang of Anthropic drew a distinction that has since become common vocabulary. Workflows orchestrate models and tools through predefined code paths; agents let the model direct its own process and tool use. They described a small set of workflow patterns, all built from an augmented LLM (a model with retrieval, tools and memory), and advised starting with the simplest design that works, often a single well-prompted call with retrieval, and adding agentic behaviour only when it demonstrably improves results.

The advice matches this course's general position on simplicity. A workflow has a known shape, so its cost, latency and failure modes can be predicted and its steps tested one by one. An agent can handle open-ended tasks whose steps cannot be known in advance, such as investigating an unfamiliar bug or researching a question across many sources, at the price of variable cost, harder testing and more ways to go wrong. Many systems described as agents would be better as workflows, and many good agents contain workflows.

Designing tools for models

Tools are an interface whose user is a model, and they deserve the care given to any API. Schluntz and Zhang call this the agent-computer interface. Give each tool one clear purpose and a name and description that say when to use it, when not to, and what it returns; models choose tools mostly from their descriptions. Keep parameters few and typed, use enumerations instead of free text, and reject unknown fields. Make mistakes hard to make, for example by requiring absolute identifiers rather than positions in a list. Return concise, relevant results, because every token of a tool result is paid for and occupies the context window; paginate and summarise large results. Return errors the model can act on (order not found; ask the customer to check the number) rather than stack traces.

Separate reading from writing. Read tools can usually run freely; write tools change the world and need validation, authorisation, idempotency and sometimes approval. Keep the number of tools available at any moment small, because large tool sets confuse models and enlarge the attack surface; route to a subset, or load tool definitions on demand. Ask for structured output whenever code will consume the result: many APIs can constrain generation to a JSON Schema, which removes parsing failures but not wrong values, so validation of the content is still needed.

The Model Context Protocol

Every assistant that wants to reach every data source or tool would otherwise need a custom integration for each pair: N applications times M systems. The Model Context Protocol (MCP), introduced by Anthropic as an open standard in November 2024, replaces that with one protocol, in the way the Language Server Protocol did for code editors. A host application (a chat assistant, an IDE, an agent) runs one MCP client per connection to an MCP server. Servers expose tools (functions the model may call), resources (data the application can read into context) and prompts (reusable templates). Messages use JSON-RPC 2.0, over standard input and output for local servers or over HTTP for remote ones, and remote servers use OAuth-based authorisation.

Adoption was fast. By December 2025, Anthropic reported more than 10,000 active public MCP servers and support in products including ChatGPT, Cursor, Gemini, Microsoft Copilot and Visual Studio Code. On 9 December 2025 Anthropic donated MCP to the newly formed Agentic AI Foundation, a directed fund under the Linux Foundation co-founded by Anthropic, Block and OpenAI, so the protocol is now governed in a vendor-neutral home. The specification is versioned by date. The 2026-07-28 revision made the core protocol stateless, so that remote servers can run behind ordinary load balancers without sticky sessions; added support for optional extensions and moved long-running tasks out of the core into an official extension; tightened authorisation; and deprecated older features (roots, sampling and logging) with a transition period of at least a year. For communication between independent agents, rather than between an application and its tools, Google introduced the Agent2Agent (A2A) protocol in April 2025; it was contributed to the Linux Foundation in June 2025, and in August 2026 its backers announced that it would move into the Agentic AI Foundation as well.

Architecturally, an MCP server is a dependency that can read your data and act on your behalf, so treat it like one. Approve and pin the servers that may be used, as part of supply-chain security; prefer servers you or a trusted vendor operate; give each server's credentials the least privilege its tools need; remember that tool descriptions and results flow into the model's context and can carry injected instructions; and route remote MCP traffic through the same gateway, logging and policy controls as other model traffic. The protocol standardises the plumbing; it does not decide what an agent should be allowed to do.

Memory and state

Models are stateless, so an agent's memory is whatever the surrounding system stores and puts back into the context. Short-term memory is the conversation and the results of tool calls so far. It grows with every step, raising cost and latency and eventually exceeding the window, so long-running agents manage it: they drop or shorten old tool results, summarise earlier turns (compaction) and keep a running plan or notes file that the model updates. Long-term memory is information kept across sessions, such as a customer's dietary preferences or facts an assistant learned about a project, stored in a database and retrieved when relevant, which makes it a small RAG system with the same questions of relevance, freshness and access control.

Execution state matters most for reliability. If an agent is halfway through a twenty-step task when its process restarts, it should resume, not start again and repeat side effects. Persist the state after each step (the context, the plan, which tool calls completed and their results) as a checkpoint, or run the agent on a durable execution engine that does this for you, in the way long-running business processes and sagas are run. Memory is also personal data: decide what is remembered, for how long, whether users can see and delete it, and keep one customer's memory out of another's context (see privacy by design).

Multi-agent systems

A multi-agent system splits work between several model-driven agents, usually a lead agent that plans and delegates to sub-agents, each with its own instructions, tools and context window, whose results the lead combines. The benefits are parallelism, specialisation and, above all, separation of context: each sub-agent can explore a large amount of material and return only a short summary. Anthropic reported in June 2025 that its multi-agent research system clearly outperformed a single agent on its internal research evaluation, but also that such systems used about 15 times as many tokens as ordinary chats, and that they suit broad, parallelisable tasks better than tightly coupled ones such as most coding.

Practitioners disagree about how far to go. In the same month Walden Yan of Cognition argued in Don't Build Multi-Agents that sub-agents working from partial context make conflicting assumptions, and recommended a single-threaded agent with careful context management as the default. Both positions agree on the underlying cause: coordination between agents is lossy communication between components that do not share state. That is the central problem of distributed systems, and the same advice applies. Do not distribute without a reason; keep the interfaces between agents explicit and narrow; prefer independent subtasks; trace every hop; and expect cost, latency and failure modes to multiply with the number of agents.

Reliability engineering for agents

Agents fail in new ways: they loop, calling the same tool with the same arguments; they wander, pursuing a side issue until the budget runs out; they give up early or claim success they did not achieve; and they take actions their designers did not anticipate. The remedies are the patterns of Chapter 9 and Chapter 6, applied at every step rather than once per request, plus a firm rule that consequential side effects need more than the model's say-so. Section 12.6 treats the security side, excessive agency, in depth; the checklist below covers reliability.