Code generation is a commodity. The defensible value isn’t “my agent writes better code.” It’s “my system can tell you whether the code is safe to ship.” That’s the layer worth building. The demo is always the generation, never the check.
The tools that exist are good at review. CodeRabbit and tools like it read a diff and comment on it, some before you commit, and GitHub’s coding agent now reviews and scans its own changes before it opens a PR. What I haven’t seen packaged is the gate that knows what the agent was assigned: did it stay in its files, did it break an invariant this codebase depends on?
That gap is where the problems live. When you point an agent at a real codebase, it defines its own boundaries of action. Tell it to improve security and it’ll make every change it considers a security improvement. Some of those changes will break your application, not because the fixes were wrong in isolation, but because the agent has no concept of why the code was written that way.
The agent optimized for what you asked. The damage comes from what you didn’t specify.
The evidence keeps piling up. Cursor turned hundreds of agents loose to generate a browser in Rust. An outside observer checked the CI: an 88% job failure rate. A Replit agent that had been covering up bugs with fake data, including a 4,000-record database of fictional people, went on to wipe the production database. Google’s 2024 DORA report estimated that as AI adoption increased, delivery stability fell 7.2%. METR gave experienced developers AI tools on their own repos. They took 19% longer while believing they were 20% faster.
The generation works. The verification hasn’t caught up.
The shape of the answer isn’t complicated. Does it compile? Did it stay within the files it was assigned? Did it break any structural invariants? Does the diff contain real changes? Run those first. Deterministic questions with binary answers. In my experience, they catch most of the problems agents introduce without burning a single token. Then, and only then, run the AI-based semantic review asking whether the changes match the original intent. Each stage is cheaper than the next. Most problems never reach the expensive stage.
Most teams skip all of this. Let the agent generate, maybe run tests, hope a human catches the rest.
An auditor can audit their own books. We don’t let them. When the same AI that wrote the code evaluates the code, it’s inclined to find its own work reasonable. A guard pipeline needs checks that are genuinely external to the generation process. Checks that don’t care how clever the solution is, only whether it meets hard criteria.
Billions going into making agents write better code. A fraction of that going into making the output trustworthy at the point of merge. As this layer matures, the generation layer becomes interchangeable overnight.