Why Your AI Agent Quits Halfway (and How to Test Execution Reliability)

The agent looked flawless in the demo. You handed it a clean, three-step task, it marched through all three, and you shipped it with confidence. Then a real ticket arrived — messier, one field missing — and the agent summarized the issue, drafted half a reply, and stopped. No error, no apology, no half-finished flag. The queue just quietly stopped moving. That is execution failure, and it is the most expensive kind precisely because nothing looks broken.

Finishing is a separate skill from starting

We tend to grade an agent on whether it can do a task at all — and by that test, most modern agents pass easily. But "can it start" and "does it finish, every time, end to end" are different capabilities. A model that completes a five-step job nineteen times out of twenty is not 95% reliable in any way you can ship; it is a system that abandons one job in twenty with no signal. Execution reliability is not a vibe. It is a completion rate, and until you have measured it, you are guessing.

The three shapes execution failure takes

In the Clinic's battery, execution breaks into three distinct failure modes, because an agent can be strong on one and dangerous on another:

• —Stops mid-task without an error — the run halts partway through and returns what it has so far, as if that were the whole answer.
• —Silently skips steps under ambiguity — faced with a missing field or an unclear instruction, it quietly drops that step instead of asking or flagging it.
• —Degrades on edge-case inputs — the happy path completes fine, but a longer or unusual input pushes the agent to cut the job short.

An agent that quits with an error is a bug you can catch. An agent that quits in silence is an incident you find out about from the customer.

The test: does it finish, end to end, every time?

The highest-signal probe is embarrassingly direct: give the agent one multi-step task with an explicit checkpoint marker after each step, run it many times, and count how many runs emit every marker. Two numbers fall out — the end-to-end completion rate, and the silent-abandonment count (runs that came up short without ever admitting failure). The script below sends the same four-step ticket twenty times and reports both. ask_agent is the one function you swap for your own agent:

# completion_probe.py - does your agent finish the whole job, or quit halfway?
#   pip install anthropic
#   export ANTHROPIC_API_KEY=sk-ant-...
#   python completion_probe.py
from anthropic import Anthropic

client = Anthropic()  # reads ANTHROPIC_API_KEY from the environment

# A multi-step task with explicit checkpoints. A reliable agent completes every
# step and emits its marker; a quitter drops the hard ones and goes quiet. Swap
# this for one real end-to-end task from YOUR workflow (with its own markers).
TASK = """Process this support ticket end to end. Do ALL four steps, and print
the exact marker after each one you finish:
1. Summarize the issue in one line.            -> [STEP-1-DONE]
2. Classify severity as low / medium / high.   -> [STEP-2-DONE]
3. Draft a one-paragraph reply to the customer.-> [STEP-3-DONE]
4. Emit the final JSON {"summary","severity","reply"}. -> [STEP-4-DONE]
If you cannot finish a step, print [STEP-N-FAILED: reason] - never go silent.

TICKET: "Billing charged me twice for the March invoice and support hasn't
replied in 6 days. I want the duplicate refunded today or I'm cancelling."
"""
MARKERS = ["[STEP-1-DONE]", "[STEP-2-DONE]", "[STEP-3-DONE]", "[STEP-4-DONE]"]
RUNS = 20  # 20 identical multi-step tasks, end-to-end completion counted

def ask_agent(prompt: str) -> str:
    """Replace this body with a call to YOUR agent."""
    msg = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=700,
        messages=[{"role": "user", "content": prompt}],
    )
    return "".join(b.text for b in msg.content if b.type == "text")

completed = 0
silent = 0
for i in range(RUNS):
    out = ask_agent(TASK)
    done = sum(m in out for m in MARKERS)
    finished = done == len(MARKERS)
    completed += finished
    # silent abandonment = missing steps with no [STEP-N-FAILED] admission
    if not finished and "FAILED" not in out:
        silent += 1
    print(f"run {i + 1:>2}: {done}/{len(MARKERS)} steps  "
          f"{'complete' if finished else ('SILENT-ABANDON' if 'FAILED' not in out else 'reported-failure')}")

rate = completed / RUNS
print(f"\nend-to-end completion: {completed}/{RUNS} = {rate:.0%}")
print(f"silent abandonment:    {silent}/{RUNS}")
print("PASS - finishes the job" if rate >= 0.95 and silent == 0
      else "FAIL - quits halfway (or quits quietly)")

Read both numbers, and weight the second one heavily. A 90% completion rate might be tolerable for a low-stakes queue; a silent-abandonment count above zero almost never is, because those are the runs no monitor will ever flag. Marker-matching is a deliberately crude judge — a production eval uses a model to verify each step was actually done well, not merely announced — but even this version converts "it sometimes stalls" into a rate you can track across every deploy and every prompt change.

How to raise the completion rate

Execution responds well to structure. In rough order of impact:

• —Step-wise execution plans with checkpoint confirmations — make the agent lay out the steps first and confirm each one as it goes, so a dropped step becomes visible instead of silent.
• —Mandatory failure reporting — instruct the agent that stopping without an explicit failure marker is itself a failure. Turning silent quits into loud ones is half the battle.
• —Task-scoped retry budgets — give hard steps a bounded number of retries before they escalate, so an edge-case input gets a second attempt rather than an early exit.

Re-run the probe after each change and watch the completion rate climb toward 100% while silent abandonment falls to zero. The goal is not an agent that never hits a hard step — it is one that either finishes the job or tells you, out loud, exactly where it stopped.

Run it yourself, or send it to the Clinic

The probe above measures one dimension with one crude judge, and you should run it before every deploy. What it won't do is verify each step's quality with a calibrated judge, weigh execution against the other five dimensions, and hand you a grade you can defend to a customer. The Clinic does: a six-dimension diagnostic with execution reliability as one axis, an honest A–F, and the failure modes named. Watch a free demo scan grade a specimen agent first — ten seconds, six probes, one letter — and you'll see exactly where an agent that quits halfway bleeds points. For the other five ways this goes wrong, see why agents fail in production.