Monitoring AI agents in production: what to watch and who

Monitoring an AI agent after launch means watching five things: whether it ran, whether the work finished, whether the output is still good, what it cost, and what changed underneath it. One named person reads a short weekly summary of those five, and the same person gets paged when a run fails. If nobody is named, you have logs, not monitoring.

This post is the working half of two earlier pieces. Who maintains your AI agents covers who owns the agent. When to retire an AI agent covers the end of its life. This one covers the long middle.

Key Takeaways

• 94% of teams with agents in production have some observability, and 71.5% have full tracing (LangChain, survey of 1,340 people, December 2025)
• Quality is still the top barrier to production, named by about one third of respondents, so “it ran” is not the same as “it worked”
• In February 2026, 5% of LLM call spans in Datadog’s customer data returned an error, and 60% of those came from rate limits
• Five signals cover most of what goes wrong. One person reading one weekly summary beats ten dashboards nobody opens

Do most teams already monitor their agents?

Most teams with agents in production already collect telemetry, and that is exactly why collecting it is no longer the hard part. LangChain’s State of Agent Engineering survey, fielded from 18 November to 2 December 2025 with 1,340 respondents, found 89% had implemented observability for their agents. Among respondents with agents in production, 94% had some form of it and 71.5% had full tracing of individual steps and tool calls (LangChain, 2025).

The same survey says quality remains the biggest barrier to production, cited by about one third of respondents. Latency came second at 20%. Only 52.4% ran offline evaluations on test sets, and 37.3% ran online evaluations.

Read those together. Nearly everyone can see what an agent did. Far fewer can say whether what it did was right. Monitoring that stops at “the process is up” misses the failure that matters most.

What are the five signals to watch?

Five signals cover the ground. Each answers one question, and each has a named place it should be read.

Signal	The question	A bad week looks like
Run health	Did it run on schedule?	A scheduled run missing, or a run that started and never reported an end
Task outcome	Did the work finish?	Runs marked complete with no output, or output nobody downstream used
Output quality	Is the work still right?	A sampled output that fails the check it passed at launch
Cost and volume	What did it spend?	Tokens per run climbing, retries piling up, a bill that moved with no change in work
Dependencies	What changed underneath it?	A model notice, a supplier file format change, an expired credential

The first two are cheap and can be automated entirely. The third needs a human, and it is the one most often skipped.

How do you watch run health and task outcome?

Run health is a heartbeat: every scheduled run writes a start and an end, and a missing end is an alert. Task outcome goes one step further and checks the result, not the process. An agent can finish a run with exit code zero and still have posted nothing.

Our own agent fleet, 41 registered agents as of July 2026, completed 5,450 runs with zero failures over a 13-day window (agent fleet proof page, 2026). That zero is only meaningful because every run is recorded and a failure is loud by design: a failed run stops, writes the reason, and notifies a named owner. A fleet where failures are quiet would also report zero, and the number would mean nothing.

The practical rule: define “done” for each agent as something checkable outside the agent. A report exists in the folder. A record appears in the system of record. A message was sent. Then monitor that artifact, not the agent’s own claim that it finished.

How do you watch quality?

Sample the output, on a schedule, with a person reading it. That is the only reliable way to catch an agent that is working and wrong.

Datadog’s State of AI Engineering report, built on LLM telemetry from over 1,000 organizations, put numbers on the loud side of the problem: in February 2026, 5% of all LLM call spans returned an error, and 60% of those errors were rate limits (Datadog, 2026). Errors like that show up in a dashboard. A plausible wrong answer does not, because the call succeeded.

So build a small habit. Each week, the owner reads a fixed sample of outputs, five to ten, against the same short checklist used at handover. Did it use the right source? Did it follow the instruction as written? Did anything surprising slip through? Keep the sample list in the repository next to the agent’s instructions, so it is versioned and so a new owner inherits it.

Where the agent produces something with a known right answer, such as a reconciliation or a calculation, add an automated check that recomputes a few cases independently. For everything else, the human sample is the control.

How do you watch cost and dependencies?

Treat spend as a signal, not a bill. A rising token count per run usually means the input grew, the instructions got longer, or retries are stacking up. Any of the three is worth a look before the invoice arrives.

OpenTelemetry’s GenAI conventions give a common vocabulary here: model name, input and output token counts, operation duration and finish reasons recorded on each call (OpenTelemetry, 2026). The conventions are still under active development, so pick the fields you will actually read and avoid building a dashboard around attributes that may be renamed. Datadog’s report also found only 28% of LLM call spans showed cached-read tokens, so caching is a common cost lever that teams leave unused.

Dependencies are the quiet one. Set a spend cap and a rate-limit alert so a loop becomes a warning rather than a bill. Then keep a short list for each agent: the model and version it runs on, that version’s retirement date if one is published, the credentials it holds and when they expire, and any supplier whose file format it parses. Review that list monthly. The maintenance post shows why: model retirements arrive on the provider’s calendar, not yours.

Who should watch, and what do they get?

One named person watches, and they should receive three things and nothing else: a failure alert the moment a run fails, a weekly written summary of the five signals, and a monthly dependency list. Everything beyond that is noise.

The summary is the product. It should fit on one screen and say, for each agent: runs, failures, outputs sampled, spend against cap, and anything pending. If it takes more than five minutes to read, people stop reading it, and the monitoring has quietly ended.

Three failure patterns recur when this is set up badly:

• Everyone gets the alert. A channel with ten people means nobody acts. One name, with a written backup.
• Alerts without a runbook. An alert that says “run failed” with no next step wastes the first twenty minutes of every incident.
• The builder is the monitor. If the person who built the agent is the only one watching, the watch ends when the engagement does.

The right owner is whoever feels the cost when the agent is wrong, usually the operations lead for the function the agent serves. They do not need to read traces. They need the summary, the alert and the authority to switch the agent off.

What does this cost to run?

The monitoring itself is small if it is designed in at build time: a heartbeat, an outcome check, a sample list, a spend cap and a weekly summary. Retrofitting it onto an agent that shipped without any of these is slower, because the “done” definition has to be written after the fact.

On the Fractional AI Officer retainer ($2,900 a month, cancel monthly), the weekly written brief covers exactly these five signals for every agent we run for you, and you read it asynchronously. Direct engagements like this are our core business. If you are an agency or software reseller, the same five signals are what you would hand to your client’s owner at delivery.

Each agent we build is a tomte, our word for one production agent with one defined job. One job means one definition of done, and that is what makes it monitorable.

Start async

Have agents running and no one reading anything about them? Send what you have at /start. You get a written reply within one business day, no meetings, naming which of the five signals your agents are missing first.