What Broke on the Way to 41 Agents: AI Agent Failure Modes

The AI agent failure modes that hurt in production are mostly not model errors. In our own fleet they were confident wrong answers, safety checks that failed in the wrong direction, state nobody could read, things living outside the register, bad plans executed well, and a budget cap that switched a lane off. Each one had a fix. Some held. Some are still open, and we say which.

This is part eight, the last part of our governance series. The previous part covered what has to be true before an agent is allowed to run. This one covers what broke anyway.

For context: as of 30 June 2026 we ran 41 registered agents across three operating companies, 9 of them live in production. Each is a tomte, our word for one production agent with one defined job. Everything below happened inside that fleet or the systems around it, between August and September 2026.

Why production failures are rarely the model

Most agent failures in production come from the layer around the model: how outputs get checked, where state is written, who owns a job, and what the system does with an input it does not recognise. The model is usually doing what it was asked. The harness is asking the wrong thing, or not checking the answer.

The research points the same way. In the September 2025 version of TheAgentCompany benchmark from Carnegie Mellon and collaborators, the best agent (Gemini 2.5 Pro) completed 30.3% of 175 simulated office tasks end to end. That is a ceiling on capability. The MAST study of multi-agent systems (Cemri et al., revised October 2025) annotated more than 1,600 traces across seven frameworks and sorted the failures into 14 modes under three headings: system design, inter-agent misalignment, and task verification. None of the three headings reads “the model was not capable enough”.

Our own list fits the first and third headings almost entirely.

Failure mode 1: confident wrong answers

The most expensive failure is a plausible answer that nobody checks against reality. It does not raise an alert, because nothing errored.

We had two. A deployment tool silently ignored a site-selection flag across a whole family of commands and read whichever site happened to be linked locally. Its listing call also capped quietly at a fraction of the real sites. A key-rotation script trusting both would have reported success while leaving most consumers on old keys. A review caught it before the script ran.

The second was our own improvement loop. Its scorecard showed zero verified fixes while real fixes had landed, because it only recognised one narrow shape of “merged” and skipped the rest without saying so.

Fix that held: verify against the live value, never against the tool’s own report. The loop now hands any ambiguous closure to a reconciler that checks whether the linked change actually merged.

Failure mode 2: safety checks that fail the wrong way

A safety check has two ways to break: it lets bad work through, or it blocks good work and floods humans. The second is quieter and more common.

A pre-push guard blocked a legitimate push because its own checker crashed on missing dependencies in a fresh workspace, and it read the crash as a finding. Earlier the same day a refactor had left that guard not installed at all on fresh clones. Separately, our escalation classifier sent every failure it did not recognise to the founder as a task. The inbox filled within days, which inverts the whole point of a human gate.

That matters beyond our fleet. Microsoft’s AI Red Team, in its June 2026 update after 12 months of testing agent systems, found human-in-the-loop bypass to be the most consistently exploited failure mode. A human gate that fires on everything trains people to click through it.

Fix that held: unrecognised failures now go to an automatic investigate lane, and only a short written list reaches a human: re-authentication, spend, policy rulings, destructive actions. The guard now checks its preconditions and skips when its tooling is absent. The honest trade-off: a skip is also a silent no-op, so the skip itself is logged.

Failure mode 3: state nobody can read

Work that stops without a reason is work nobody can triage. An agent can park a job correctly and still leave it unrecoverable if the record says nothing about why.

An audit of parked work in September 2026 found a large share with a blank hold reason, and a set of plain infrastructure hiccups that nothing ever re-queued. Separately, a daily publishing routine that passed no run key left an orphaned lock. Fixing that by registering one owner per publishing slot surfaced a real case where two agents legitimately shared one.

Fix that held: one helper now writes every hold reason, and a test fails if any new code bypasses it. Automatic retry works from an allow-list, fails closed on anything unrecognised, and ships switched off by default. Still open: the slot-ownership gate runs in shadow mode, logging violations without blocking, until the shared-owner case is designed properly.

Failure mode 4: things outside the register

Anything the register does not know about will be missed by every tool that reads the register. We learned this with credentials, not agents.

A secret was changed outside our rotation tooling. Two local copies on staff machines went stale, and a downstream content sync rejected every call for hours until a human noticed. The same root cause had hit two other credentials that week: consumers nobody had registered, so the tooling never reached them. OWASP’s Top 10 for Agentic Applications, published 9 December 2025, lists identity and privilege abuse third for the same reason: access that nobody inventoried is access nobody rotates.

Fix that held: every consumer of every secret is now in the credential manifest, local files included. Rotation is atomic, so a failed verification probe rolls everything back, and a scheduled drift check catches changes made outside the tooling.

Failure mode 5: bad plans, executed well

An agent that executes a flawed plan perfectly still ships a flaw. Checks on the output miss it, because the output matches the plan.

An audit of our content desk’s defect tasks in August 2026 found briefs that named products we do not sell, used an out-of-stock item as the call to action, repeated the same topic across briefs, and made provenance claims nobody could verify. The writing agents did their jobs. The plans were wrong.

Fix, partly held: a validation step now hard-blocks a brief before it becomes a card, with cadence checks for repeated topics and anchors. Still open: the planner side of that wiring was on the to-do list at the time of writing.

Failure mode 6: the budget is a failure mode too

Cost caps switch things off, and a switched-off agent looks exactly like a quiet day. Plan for it the way you plan for an outage.

In September 2026 we hit the monthly limit on our build minutes. The cloud lane that lets agents propose code fixes was paused by commenting out its schedule, as an authorised one-line revert. A free local lane still drains the queue when that machine is on. Still open: the fallback that should resume the cloud lane automatically when the local one goes down is designed but not switched back on.

The six at a glance

Failure mode	How we found it	Fix	Held?
Confident wrong answer	Pre-run review, founder question	Verify against live state	Yes
Safety check fails wrong way	Blocked push, flooded inbox	Investigate lane, short human list	Yes
Unreadable state	Audit of parked work	One writer for reasons, fail-closed retry	Partly
Outside the register	A human noticed	Every consumer registered, atomic rotation	Yes
Bad plan, good execution	Defect audit	Pre-card validation	Partly
Budget cap	Usage alert	Local lane, fallback designed	No

What the fixes that held have in common

Three habits show up in every fix that held. Check against live state, never against a report. Fail closed on anything unrecognised, but route it to a lane rather than a person. Give every field exactly one writer.

Watching is not the same as checking. LangChain’s State of Agent Engineering, a vendor survey of 1,340 builders fielded in late 2025 and published June 2026, found 89% have observability on their agents but only 37.3% run online evaluations. We were in that gap for most of these six. Part one of this series described the register and the run logs. The logs showed all six failures. None of them fired an alert on their own.

What this costs

For a client, this kind of failure review is part of the Fractional AI Officer retainer at $2,900 a month, cancel monthly. We audit what your agents claim against what actually happened, write the fixes, and hand over the checks, the runbooks, and the failure log itself. You own all of it.

Our dated fleet figures, including 5,450 runs in one 13-day stretch with zero failures, are on our own fleet’s proof page. That number and this post are both true. Zero failed runs does not mean nothing broke.

If you already run agents and suspect one of these six is happening quietly, start async. Describe what runs unattended and what it reports, in writing, and we reply within one business day. No calls.