Why AI Agents Fail in Production: The Six Failure Modes, and How to Fix Each

Search for why AI agents fail in production and the first screen hands you a number before it hands you an idea. Seventy percent. Ninety-five. Forty percent of agentic projects cancelled by 2027. Read them closely and they are all the same shape — a survey about somebody else's agents, quoted by a vendor who would like to sell you the fix. Not one of them tells you which half yours is in. If you ship on Thursday, that is the only number that matters.

Reliability multiplies. It does not average.

Here is the arithmetic that belongs at the top of every one of those articles, and it costs you nothing to check. Say one step of your agent works 70% of the time. Most teams would call that fine. It looks healthy on a dashboard and it survives a demo. Chain three of those steps and the whole run succeeds 0.7 × 0.7 × 0.7 = 34% of the time. Ten steps and you are at 2.8%. The per-step rate is the one you measure. The chain rate is the one your user gets. They are not the same number, and the gap widens with every tool you bolt on.

Which is why "it mostly works" is not a status report. It is a per-step number wearing a whole-run costume.

A 95% agent sounds like an A. Chain five of them and nearly a quarter of your runs still fail.

The six ways it actually breaks

Failure is not one condition with one cure. In our battery it is six independent axes, and an agent can be genuinely excellent on five of them while the sixth quietly costs you a customer a week. They are worth naming separately because the fixes have nothing to do with each other — and because a failure mode you cannot name is one you cannot search for at two in the morning.

• —D1 Truthfulness — it states the thing it cannot know, in exactly the confident register it uses for the things it can. The expensive version is not a wrong answer; it is a correctly formatted one. See why your agent invents facts, what to do when it lies, and the truthfulness dimension.
• —D2 Execution — it announces the job is finished with step three skipped. This is the most commonly failing axis in our battery and the hardest to catch by reading, because the transcript says done. See why your agent quits halfway and the execution dimension.
• —D3 Consistency — same input, different answer, no error either time. Run one case twice and diff the outputs. Guess first whether they will match; the guess is the interesting part. See why your agent gives different answers and the consistency dimension.
• —D4 Tool use — right intent, wrong call. A hallucinated parameter, a malformed payload, or an API that returns something odd and gets improvised around instead of raising. See why your agent calls the wrong tool and the tool-use dimension.
• —D5 Context — the rule you gave it in turn one is gone by turn forty and nothing announced the loss. Attention thins across accumulated tool output, and the middle of a long context goes first. See why your agent forgets the rule you gave it and the context dimension.
• —D6 Recovery — the tool 502s and the agent reports a successful retry that never happened. The one axis a transcript cannot prove, because you have to break the world on purpose to see it. See why your agent says it recovered and the recovery dimension.

Get your own rate before you fix anything

Every fix below is cheap to apply and impossible to evaluate without a baseline, so measure first. The probe is deliberately dumb: one task you care about, run twenty times, graded by a check a machine can make without opinions. Then it prints what that rate becomes over a chain. Keep the grader mechanical — the moment a human has to squint at an output and decide, you are collecting a mood, not a rate.

"""reliability_rate.py — measure YOUR agent's per-run success rate, then see
what that rate is worth over a chain.  pip install anthropic
"""
import json
import os

from anthropic import Anthropic

client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
MODEL = "claude-opus-4-8"

# One task you actually care about, and a check a machine can make. If a human
# has to squint at the output to call it, you are collecting a mood, not a rate.
TASK = 'Total these line items. Reply with only JSON {"total": <number>}: 19.99, 4.50, 120.00'
EXPECT = 144.49
N = 20


def run_once() -> bool:
    r = client.messages.create(
        model=MODEL,
        max_tokens=256,
        messages=[{"role": "user", "content": TASK}],
    )
    text = "".join(b.text for b in r.content if b.type == "text")
    try:
        got = json.loads(text[text.index("{"): text.rindex("}") + 1])["total"]
    except Exception:
        return False  # unparseable IS a failure. Do not hand-repair it and re-score.
    return abs(float(got) - EXPECT) < 0.005


passes = sum(run_once() for _ in range(N))
p = passes / N
print(f"per-run success: {passes}/{N} = {p:.1%}")
for steps in (1, 3, 5, 10):
    print(f"  {steps:2}-step chain, every step must hold: {p ** steps:.1%}")

Two things about that script matter more than the script. Unparseable output counts as a failure — if you repair it by hand before scoring, you have measured yourself, not the agent. And twenty runs on one task buys you a rough rate, not a precise one. That is fine. The point is to learn whether you live at 0.99 or at 0.7, because those two are indistinguishable in a demo and live in different worlds five steps out.

Fix in the order the failures hide each other

The instinct is to fix the loudest one first. The better order is to fix the ones that wreck your ability to see the rest. D1 and D6 come first, because an agent that misreports what it did takes your instrumentation down with it, and every measurement you make afterwards is downstream of a witness with an interest in the verdict. Then D4, since a malformed tool call poisons the context that D5 and D2 are running on. Consistency last. It is real, but an agent that stably does the wrong thing is not the emergency in the room.

We know that order from the wrong side of it. Our own grader once gave a fabricated $150 refund a perfect 100 on truthfulness — the scorer, whose entire job is catching invention, scored the invention full marks. We shipped the fix that morning. It is the best argument we have that the battery works: the first thing it caught was us.

Run it yourself, or send it to the Clinic

Everything above is yours to run without us, and the probe is the whole method — there is no step held back. What it will not do is weigh six axes against each other, run the adversarial passes, or hand you a grade you can put in front of a customer. That is what the Clinic is: an 18-test, six-dimension diagnostic with a temperature-0 judge, an honest A–F, and any dimension your sample could not evidence marked NOT TESTED and excluded rather than guessed. Five scans a month are free. If you want the methodology before the tool, how to test an AI agent lays out the framework and a free scan runs it. If the answer turns out to be that someone else should do the repair, post the work — executors bid inside your budget, and they clear this same battery before they are allowed to bid at all.