PHASE 04 · AGENT
Goal: Move from one-shot Q&A to an LLM that decides actions.
An agent is an LLM that can call tools (functions you define) and decide what to do next. You hand it a list of available tools with descriptions and parameter schemas. Instead of replying with text, the model can reply with "call search_docs(query='...')". Your code runs the tool, feeds the result back, and the model continues — maybe calling more tools, maybe answering. This think-act-observe loop is the agent.
The key shift from a workflow to an agent: in a workflow, you decide the order of steps. In an agent, the LLM decides. That's powerful (handles novel inputs) and dangerous (the LLM picks badly when tool descriptions are vague).
Tool descriptions are secretly another prompt. A good description tells the model when to use the tool, what it does, and what arguments mean. Bad descriptions cause the model to pick the wrong tool or hallucinate arguments — the #1 cause of agent failure.
tool_use, run the tool and append the result; otherwise, return the final answer. Cap at ~10 iterations to prevent runaway loops.search_docs tool.agent.py — A chatbot that routes questions to the right tool: RAG, web search, or calculator.
Concepts unlocked: function calling · the agent loop · ReAct · tool descriptions
Everything so far has the model talking — it reads text and writes text back. But it can't do anything: it can't look something up, run a calculation, or check today's date. Ask a plain model 8,347 × 219 and it'll hand you a confident, wrong number.
The obvious fix: give it a calculator. But here's the part almost everyone gets backwards at first — the model itself never touches the calculator. Remember Phase 1: an LLM is a math function, reached over an API, running on someone else's computers. It has no hands, no internet connection, no ability to run code. All it can ever do is send back more text.
Here's a picture worth keeping for the rest of the course: the LLM is a brain in a jar. Brilliant, endlessly well-read — but sealed behind glass. It can hear you and talk back, and that is the entire list of things it can do. To get anything done in the real world, it needs a pair of hands outside the jar: something that listens for “please press the calculator buttons for me,” actually presses them, and reads the display back through the glass. This whole phase is about building those hands.
So what does “give it a calculator” actually look like? Just one small round trip:
Notice what the LLM actually contributed: not the answer 1,828,993 — just the numbers and the operator, a=8347, b=219, op=×. That's still just a prediction, the same next-word mechanism from Phase 1 — it's predicting which tool-call text comes next, not computing anything. The real multiplication happens somewhere else entirely, which is exactly what the rest of this phase is about.
The critical correction. The LLM never executes a tool. It can only reply with text saying which tool it wants and what arguments to use. Something else has to actually run it and report back — code that runs on your side, called the harness. (And no, you won't be writing that code by hand — your coding assistant writes it for you, in the Build section.) Get this backwards and nothing else in this phase will make sense.
In the intuition we left the LLM as a brain in a jar, asking for a pair of hands. The harness is those hands. Concretely, it's just a program: it holds the real tool functions (an actual calculator, an actual web search), sends the LLM a request, reads what it replies, and — when the LLM asks for a tool — is the one that actually runs it.
And here's where those hands live: on your side. The harness runs wherever you run your program — your laptop today, maybe a server you control later. The model, meanwhile, stays exactly where it's been since Phase 1: behind the API, on the provider's computers. Two different machines, talking over the internet. Nothing about the model moved; you just put smarter code on your end of the line.
One more detail that trips people up: the LLM never sees your real function. It only ever sees a description — the tool's name, what it does, and what arguments it takes. That description is what you hand the model as plain text; the actual working code never leaves your side. It stays inside the harness.
Step 3 said “the loop runs the real function” — but what actually counts as a tool here? Almost anything. A tool isn't one specific kind of thing; it's just some real action your code can perform, wrapped up with a name and a description so the LLM knows it's available.
calculator(a, b): pure code, no network, just arithmetic.get_weather(city): reaches out to a weather service over the internet and returns whatever it says.run_sql(query): runs SQL against your company's database and returns the rows. (This is exactly Phase 5's tool.)read_file(path): opens a file sitting on your machine and returns its text.search_docs(query): runs the RAG pipeline you built in Phase 2.send_email(to, subject, body), create_calendar_event(...): doesn't just return information, it actually changes something.The LLM can't tell the difference. From the model's side, every tool looks identical — a name, a description, some arguments. Whether calling it means doing harmless arithmetic or sending a real email to a real customer is entirely up to what your code does when the harness runs it. Guard the risky ones accordingly (more on this in Phase 7).
Same shape as Phase 1's loop. This is still the predict-and-continue loop from Phase 1. The only difference: some of the "words" the model can produce are tool calls instead of plain text — and now there's a harness in the middle, carrying messages back and forth and doing the actual work.
Put all of that on repeat and you get the full agent loop. Watch it happen, start to finish, on the calculator example:
Notice the harness does all the moving — it carries the question to the LLM, carries the decision back, runs the real tool, carries the result back, and finally carries the answer out. The LLM only ever sits in one place and decides.
Did you catch “one or more”? The model can ask for several tools in a single reply when they don't depend on each other. Ask “which is warmer today, Boston or Seattle?” and it may request get_weather("Boston") and get_weather("Seattle") in the same turn — the harness simply runs both and sends both results back together. When one call depends on another's result, though, it has to go one round at a time — you'll see exactly that in the next section.
Always cap the loop. Cap it at some number of rounds (say, 10) so a confused agent that keeps asking for tools can't spin forever. Enforcing that cap is entirely the harness's job — the model has no concept of "too many rounds."
Workflow vs agent. In a workflow, you hard-code the order of steps. In an agent, the model decides the order — the harness just carries out whatever it asks for. Agents handle surprises a fixed script can't — but they also fail in ways a fixed script can't. Reach for an agent only when you genuinely can't lay out the steps ahead of time.
One round trip is enough for a calculator. Real questions usually aren't that polite. Here's one from Zentara Logistics — our made-up shipping company — that no single tool can answer: “Where is order #4817 right now?”
The harness has two tools on offer: lookup_order(order_id) — fetches an order's status and which truck it's on — and get_truck_location(truck_id) — returns a truck's live GPS position. Neither alone gets you there. Watch the loop chain them:
Two things to notice. First, the second tool call used the first one's result — the model couldn't have asked for T-42's location before learning the truck was T-42. That's why chained calls go one round at a time, unlike the two-cities weather example. Second, nobody scripted that order. There's no line of code anywhere saying “first look up the order, then check the truck.” The model read each result and decided what it needed next.
This is what "agent" actually means. The model plans the route — which tool, with what arguments, based on what it just learned — and the harness does every step of the actual walking. Decision-making and doing are split across two machines: the model behind the API decides; your code at home does.
In the Zentara example, how did the model know lookup_order even existed, or what to pass it? Only from its description. That's why the single biggest thing that makes or breaks an agent isn't the model — it's how you describe your tools to it. A tool description is really just another prompt: it's the only information the LLM has when deciding whether to reach for that tool.
Good news: a good description follows one simple rule. Every time, it answers three questions:
Watch the rule pass and fail on the same tool:
This is the #1 cause of agent failure. Vague tool descriptions make the model pick the wrong tool or make up arguments. Spend your effort writing clear what / when / arguments descriptions — it helps more than a bigger model.
Time to build agent.py — a harness that routes each question to the right tool: your Phase 2 RAG for document questions, a web search for current events, a calculator for math. You'll have the loop written plainly (no framework yet) so you can see every moving part.
Read what it builds. The heart of it is the loop — the harness keeps running tools until the model stops asking for them. (The sample below uses Anthropic's SDK, the course default. If you picked OpenAI back in Phase 1, your assistant will produce the equivalent — the names differ, the shape of the loop is identical.)
# agent.py — the harness: carries messages, runs tools, never lets the model touch either
from anthropic import Anthropic
client = Anthropic()
TOOLS = [search_docs_def, web_search_def, calculator_def] # descriptions only — name + schema
def run(question):
messages = [{"role": "user", "content": question}]
for _ in range(10): # the harness caps the loop
reply = client.messages.create(
model="claude-sonnet-4-6", max_tokens=1024,
tools=TOOLS, messages=messages)
messages.append({"role": "assistant", "content": reply.content})
calls = [b for b in reply.content if b.type == "tool_use"]
if not calls: # no tool call -> this IS the final answer
return reply.content[0].text
results = [
{"type": "tool_result", "tool_use_id": c.id,
"content": run_tool(c.name, c.input)} # the harness runs it — never the model
for c in calls
]
messages.append({"role": "user", "content": results})
return "Stopped: too many tool calls."Spot the plural. See the variable calls? Plural on purpose. That's the several-tools-at-once case from the agent-loop section: if the model asks for two tools in one reply, the harness runs each real function and sends all the results back together.
Deliverable — agent.py. A harness that routes each question to the right tool — RAG, web search, or calculator — runs it for real, and loops until the model can answer. Phase 5 hands it one more tool: your database.