Every agent demo can succeed once. The interesting question is the fifth run: the input is real, the tools are slow, and the user doesn't care about the architecture.
Most of the failures I see aren't reasoning failures, they're loop failures: still searching after the answer is found; calling a tool because it's available; editing a file without reading it; summarizing without verifying; asking permission only after doing the irreversible thing. The model is often capable. The loop has no taste.
The fifth run
The first run is a rehearsal. The prompt is clean, the tools answer instantly, the happy path is obvious. The fifth run has a malformed input, a flaky API, a stale cache, and a user who wanted the result yesterday. That's where the design lives.
I care about agents that can be interrupted, inspected and resumed. Not because interruption is elegant, but because real work gets interrupted anyway. A loop that can't survive a tool pause is a script with an LLM attached.
A demo proves the model can act. A product proves the loop can stop.
The tool-belt trap
Adding tools feels like adding capability. In practice every tool adds a branch, a schema, a failure mode, and a new way to be confidently wrong. The search space grows faster than the useful behavior. An agent with twenty tools is often worse on a focused task than one with three tools and one clear instruction.
The tool belt also hides the real bottleneck: context. A long list of tool descriptions competes with the task, the history, retrieved documents and constraints. The model spends more attention deciding which tool to use than on whether the task is already done.
My default is to start from the smallest set that can close the loop: one read tool, one write tool, one way to ask a human. Add the fourth when a trace proves the loop can't move without it.
The loop is the product
A useful agent loop has the same parts as any other product interface:
- Entry state. What does the agent know before it starts? What's missing? What must it never assume?
- Observation. What counts as evidence? A tool result, a screenshot, a passing test, a human confirmation?
- Action. What can change the world? Which parts are reversible? Which need a check?
- Exit. What does done look like? Who decides? What happens when the budget runs out?
Most teams write the prompt and skip the exit. Then they're surprised the agent never stops. The exit isn't a guardrail bolted on at the end. It's the spec.
Before you add a tool, write down the condition under which the loop stops. If you can't write it, you don't have a task yet, only exploration.
Taste is the stop function
An agent's taste isn't a mysterious aesthetic gift. It's a set of judgments about “enough”: this evidence is sufficient, this result is good enough, this path isn't worth another token. It's the ability to stop at the right moment, and to tell “done” apart from “budget exhausted”.
That judgment can be encoded, at least in part. Three mechanisms do most of the work:
- Budget. Caps on tokens, time, attempts and tool calls. When the cap hits, the agent reports what it has and what it didn't do. It doesn't loop silently.
- Success check. Tests, schemas, diffs, citations or human confirmation. Something outside the model decides whether the loop is finished.
- Loop detection. Repeated actions, repeated searches, or two rounds with no state change. Stop and explain, instead of retrying forever.
None of this needs a smarter model. It needs an editor's sense of when the scene ends.
What to measure
Agent evals are often reported as one success rate. That number hides the interesting part. I want to see:
- Success on a fixed task. Same input every time, with a clear pass condition.
- Cost per success. Tokens, tool calls, wall-clock time. A cheap failure is still a failure; an expensive success is a design smell.
- Failure modes. Wrong answer, loop, stopped early, unsafe action, unreadable output. Each one points at a different fix.
- Recovery. After a tool error or a human correction, can the agent continue without starting from scratch?
- Trace readability. Can a human explain what happened in two minutes? If not, the loop isn't observable enough, or improvable enough.
A trace isn't a log dump. It's a narrative: what the agent believed, what it tried, what changed, why it stopped. If the trace can't carry that narrative, the agent isn't ready for anything that matters.
Handoff
The most important tool for a serious agent is handoff. The loop says: “I did what I could; here's the state, here's the uncertainty, here's what I need from you.” That isn't a failure of autonomy. It's the feature that makes autonomy safe enough to use.
Good handoffs are specific. Not “something went wrong” but “the deploy failed at step 3; the migration ran, but the app is still on the old image; I didn't touch production traffic.” A human can act immediately. Bad handoffs dump a stack trace and call it transparency.
I design the handoff first, then remove it where the loop earns trust. Order matters. Trust is granted on evidence, not on a demo.
Checklist
- One task, one pass condition. Write both before the prompt.
- Fewest tools. Add capability only when a trace proves it's missing.
- Hard budgets. Tokens, time, attempts, tool calls. Make the cap visible to the agent.
- State before action. Read before write; snapshot before migrate; ask before irreversible.
- Stop on repetition. If the state didn't change, don't try it the same way again.
- Show humans the trace. A readable narrative, not raw logs.
- Clean handoff. State, uncertainty, next step.
Models will keep getting better, and the loop will still need an editor. The difference between an agent that feels smart and one that feels exhausting is rarely the model. It's the taste that knows when the work is over.
Context / Notes from building tool-using agents and verification harnesses, including the six-gate Photoreal Game Harness. No model benchmark is claimed here; the argument is about loop design and eval practice.