Testing a coding agent
The dangerous coding-agent failure is not a syntax error, which the compiler catches for free. It is the run that reports success on a change that never landed, because nothing in the loop checked whether the write actually happened.
Where coding agents break first
Multi-step edits are the job, so silent abandonment mid-refactor is the first failure; wrong tool calls and a failure it papers over are how it goes unnoticed.
- Stops mid-task without an error
- silently skips steps under ambiguity
- degrades on edge-case inputs.
- Calls the wrong tool confidently
- hallucinates parameters
- ignores tool output and answers from prior.
- Retries the same failing call forever
- apologizes and abandons
- hallucinates success after failure.
Nine probes to run before this one goes near a customer
How the Clinic fixes it
A grade is not a hire decision
The battery grades capability. It does not know your job. An agent can pass all 18 tests and still be wrong for the work — which is why the Hire Check runs your own task, finish the change it was given and say so honestly when it did not, and returns a separate verdict.
Where your samples do not cover a dimension we return NOT TESTED rather than a number. An untested axis reported as a score is the failure this whole product exists to avoid.
// The Clinic's live simulator ships a scripted profile for this archetype (AGT-CODE-03), whose weakest axis is D3 Output Consistency. That profile is a scripted demo, not a measurement of your agent — run the demo scan for a real one.
How do you test a coding AI agent?
Give it a multi-step change and verify the end state independently of what it reports. Check tool arguments against the real schema, and inject a failing build to see whether it recovers, escalates, or claims success anyway.
Grade yours, not an archetype
The free scan runs the real battery against your agent and returns an honest A–F grade with the failure modes named. In 24 hours you go from guessing to knowing.