PHASE 06 · OBSERVABILITY
Goal: Instrument the agent so you can debug and optimize it.
Once your agent makes multiple LLM and tool calls per request, you can't debug it from the final output alone. You need traces — a structured timeline of every step: which prompt went in, what came out, how many tokens, how long it took, what it cost.
Tools like Langfuse (cloud, easy) and Arize Phoenix (local, open-source) wrap your LLM SDK calls and ship spans to a UI. You can click any request and walk through every model call and tool invocation in order. This turns "why did it answer wrong?" from hours of guessing into a 30-second scrub.
Once you have traces, you also have aggregate metrics: avg/p95 latency, tokens per request, $/request. Now you can optimize: enable prompt caching, route simple subtasks to a smaller model, trim retrieved context, ask for structured JSON output instead of parsing prose. A 30–50% cost cut without quality loss is typical.
traces.ipynb — A dashboard showing per-request traces and aggregate metrics (latency, tokens, $).
Concepts unlocked: tracing · prompt caching · token economics · model routing
In the last phase you gave your agent a second tool — it can now answer from documents and from your database. Which means one question can trigger several calls behind the scenes: a doc search, a SQL query, the model thinking two or three times. When the final answer comes back wrong, it tells you almost nothing about why. Which of those steps went sideways? You can't see it.
What you need is for every request to keep a diary: each step writes an entry — what went in, what came out, how long it took (the wait time — engineers call it latency), and what it cost. That diary has a name: a trace.
Watch one request write its diary. The chip up top is the request traveling through your agent; the timeline underneath is the trace filling in, one entry per step:
Notice what the timeline gives you that the final answer never could: you can see which step was slow, which step cost money, and what each step actually did. Once every request leaves a diary like this, debugging stops being detective work.
A trace is the request's diary. A trace is a step-by-step timeline of one request — every model call and tool call, with its inputs, outputs, time, and cost. It turns "why did it do that?" from guesswork into reading.
Each entry in the diary has a technical name: a span — one model call or one tool call, with its details. Stack the spans in order and you get the full trace tree for that request. Tools like Langfuse (hosted — it runs on their servers, you just sign up) or Arize Phoenix (runs on your own laptop) record the spans for you and show the tree in a UI you can click through.
Make it concrete — here's what one real request's trace tree actually looks like, spans in order:
Without a trace, all you'd see is that final answer, 1.37 seconds later. With one, you can see exactly where the time went — here, almost all of it is the last LLM call, not the search. That's the difference between guessing at a fix and reading the actual bottleneck.
You don't write traces by hand. These tools give you a wrapper — a line that sits around your model calls and records everything automatically. You add it once and the timeline shows up on its own.
You've seen what a healthy trace looks like. Now use one to catch a real bug — this is the moment traces earn their keep.
A Zentara employee asks the agent: "How much parental leave do I get?" It answers, confidently: "Zentara offers 15 days of paid vacation per year." Fluent, polite — and about the wrong policy entirely.
Your first instinct is to blame the prompt: "maybe I should tell it to be more careful about leave types…" You could spend an evening rewording instructions. Instead, open the trace:
So the fix is in retrieval — better chunking, a better search query, maybe smaller chunks per Phase 2 — not in the prompt. A rewritten prompt would have changed nothing, because the model was never the problem. Thirty seconds of reading saved you an evening of tuning the innocent step.
Fix the step, not the symptom. A wrong answer only tells you that something failed. The trace tells you which step failed. Without it, you tune whatever step you can see — usually the prompt — while the guilty one stays broken.
Traces don't just catch bugs — once every call is recorded, you also get the numbers. Start with money. Zentara's agent averages $0.006 per question. Sounds like nothing — until 500 employees ask it 4 questions a day: 2,000 requests × $0.006 = $12 a day, about $360 a month. Small numbers multiply.
Time gets measured too. Alongside the average latency you'll see p95 latency everywhere, so here's what it means: line up all your response times from fastest to slowest — p95 is the time that 95% of requests beat. In plain words: almost everyone waits less than this. It's your bad-day number, and it matters more than the average, because users remember the slow requests, not the typical ones.
And once you can measure cost, you can cut it — usually 30–50% with no quality loss. On Zentara's $360/month, that's $110–180 back:
You can't optimize what you can't measure. This is why observability comes before tuning — the trace shows you exactly where the time and money are going, so you fix the real bottleneck instead of guessing.
Add tracing to the agent you've been building, then read the numbers. You'll use Langfuse (free hosted tier) or Phoenix (local). One heads-up before the code will do anything: Langfuse needs a free account and two API keys (a public one and a secret one) so it knows where to send your traces. That's a five-minute signup at cloud.langfuse.com — and the prompt below asks your assistant to walk you through it first. Phoenix skips the signup entirely because it runs on your machine.
Read what it sets up — the key idea is a wrapper that records each call automatically:
# needs your Langfuse keys in .env first (free account — your assistant
# walks you through it). Then: wrap the client once, every call is traced.
from langfuse.anthropic import Anthropic # drop-in wrapper, auto-traces calls
client = Anthropic()
client.messages.create(
model="claude-sonnet-4-6", max_tokens=512,
messages=[{"role": "user", "content": question}])
# recorded automatically: prompt, response, tokens, latency, cost
# later: read the aggregate numbers, or open the Langfuse UI
# avg latency · p95 latency · tokens/request · $/requestDeliverable — traces.ipynb. A dashboard of per-request traces plus the aggregate numbers (latency, tokens, $). Now you can debug by reading, and cut cost by 30–50% on purpose instead of by luck.