The Agent Doesn't Fail by Being Wrong. It Fails by Passing Your Test.

TL;DR: The failure mode of AI coding agents that should worry you isn’t hallucination - it’s a correct answer to a badly-framed question. An agent optimises the signal you show it: a green build, a resolved error, a passing test. It has no view of the signals you didn’t show it - build time at scale, the right base image, whether the reviewer is another agent. This post pulls together three threads from my notes: the minimalism-vs-features split (Pi vs Claude Code), agents as code reviewers, and the containers case where the model’s advice was right and the outcome was wrong.

---

Most writing about where AI coding agents fail reaches for hallucination - the confidently invented API, the function that doesn’t exist. That’s real, and it’s also the easy failure, because it’s visible. The failure that actually costs you is quieter, and I keep running into the same shape of it. Let me start with the cleanest example.

A correct answer to the wrong question

Here’s a case I wrote up in more detail in the container-images post, boiled down to its spine:

1. Your Alpine-based build fails because a dependency has no musl binary.
2. You paste the error into the model.
3. It replies apk add build-base - and it works. The build passes.
4. The error is gone, and a 15×-slower build is now baked into hundreds of pipelines.

That is not a hallucination. The advice is formally correct - adding the build toolchain does fix the error. The model was handed an error message and it removed the error message. What it was never given was the context that the right decision is to change the base image, because the question you asked was “fix this error,” not “is this base image right?”

This is the whole thing in miniature: the agent optimises the signal you show it. A green build is an easy signal to optimise. Build time multiplied across your CI fleet, the cloud bill, the correctness of the base image - those aren’t in the context, so they don’t get optimised. The risk isn’t the agent that’s wrong. It’s the agent that passes exactly the test you set it, while the thing you actually cared about quietly degrades.

The minimalism argument: how much agent do you even want?

If agents optimise the signals in their context, then the amount of machinery around them is itself a variable worth questioning - and there’s a genuine schism in the tooling about it.

The mainstream agents - Claude Code, Open Code - keep adding surface: MCP servers, sub-agents, plan mode, background bash, permission gates. The counter-position comes from Pi, a deliberately minimal agent by Mario Zechner (of libGDX fame). His argument, roughly: users touch maybe 5% of the available features, and the rest is cognitive load and context footprint without a proportional return. So Pi ships as little as possible and makes extensibility the priority - no MCP, no sub-agents, no plan mode by default, each addable as a package only if you want it. It’s model-agnostic (Claude, OpenAI, Ollama, local models), keeps a tiny system prompt, and its one standout feature is fork sessions - branching a conversation from any message to avoid contaminating context with side-quests.

The line I keep coming back to: “all these agents are just a bunch of loops on top of [models].” Strip the branding and an agent is an abstraction layer over a model, where the value lives in ergonomics and extensibility, not in built-in features.

But the minimalism case has its own asterisks, and it’s only fair to state them:

• Does minimalism scale? The very person praising Pi admits going back to a feature-rich agent for advanced work. Pi is a “keeper” for lighter tasks and portability (a home server, a quick one-shot query), not a full replacement.
• A community package ecosystem is not a vetted one. “Add it as a package if you want it” trades built-in-and-tested for bring-your-own-and-trust.
• The minimalism is a little ironic. Pi is written in TypeScript and needs an npm/JS runtime - a “minimal” agent that drags in Node. Rust or Go would fit the philosophy better.

I don’t think the schism resolves cleanly, and I’m not sure it should. The useful takeaway isn’t “pick the small one” - it’s that every feature is context you’re spending, and context is the scarce resource that determines quality. More machinery isn’t free.

Agents as reviewers - and AI reviewing AI

The other place agents are moving into is code review, and that’s where the “passes your test” problem gets recursive.

The pattern (from a terminal review tool called Hunk): one agent session loads a review skill, reads the diff, and leaves line-level notes - “non-structured response body in this endpoint” - right alongside a human’s notes. A second agent session lists those comments, loads the context, and fixes them. Run it under a “babysitter” for quality and you have a review loop that barely involves a person.

It works. It also raises the question the source itself asks, and I think it’s the right question: in an era of AI-generated slop, is the goal still “fix the comments,” or is it “be the quality gate”? If one agent writes the code, another agent reviews it, and a third fixes the review comments, where does a human enter - and if they don’t, haven’t you built an echo chamber that passes its own tests by construction?

This is the same failure mode as apk add build-base, one level up. An agent reviewing an agent optimises for “review comments resolved.” Whether the code is good - whether it should exist, whether the design is right - is a signal nobody put in the loop. The green checkmark is easy; the judgment is not.

What I actually do with this

I’m not anti-agent - I use them daily, including to draft and cross-check the notes these posts come from. But I’ve settled into a few rules that all come back to the same principle:

• Ask the question you mean. “Fix this error” gets you the error gone. “Is this the right base image?” gets you a decision. The framing is the work.
• Keep a human on the signals the agent can’t see. Cost at scale, architectural fit, whether something should exist - these aren’t in any context window unless you put them there, and often you can’t.
• Treat every feature as spent context. The minimalists are right that machinery isn’t free, even if they overstate the case.
• Don’t let agents close the loop on their own quality. AI reviewing AI is a fine accelerator and a bad final judge.

The one-sentence version: an agent will reliably give you what you asked for, which is exactly the problem. The skill that’s actually scarce now isn’t getting an agent to answer - it’s knowing what to ask, and noticing when a passing test is measuring the wrong thing.