AI Operations Dashboard: What a Company Run by Agents Watches

An AI operations dashboard is the one place where every agent, every run it completed, and every task it produced are recorded, owned and reported. It is not a wall of charts. If it cannot tell you which agent did what, when, on whose instruction, and whether a person accepted the output, it is decoration with a refresh button.

We built ours because we needed it, not because it demos well. As of 2026-06-30 our group ran 41 registered agents, 9 live in production and 5 in beta. In the 13 days measured to 2026-07-04 those agents completed 5,450 runs with zero failures. Nobody holds that shape in their head, and nobody should try.

Why the dashboard problem arrived this year

Gartner’s April 2026 press release on agent sprawl predicts the average global Fortune 500 enterprise will have over 150,000 agents in use by 2028, up from fewer than 15 in 2025. That is not a tooling problem arriving slowly. It is an inventory problem arriving all at once.

Deloitte’s State of AI in the Enterprise, published April 2026 from 3,235 IT and business leaders across 24 countries, found that 21% of organisations have a mature governance model for agentic AI. The capabilities Deloitte names as missing are exactly the three a cockpit exists to supply: clear boundaries on which decisions an agent makes alone, monitoring that flags odd behaviour while it is happening, and audit trails covering the full chain of actions.

The registry comes before the charts

The first screen is a list, not a graph. Every agent in the group is registered before it is allowed to run: what it is called internally, which system it touches, which company it belongs to, who owns it in human terms, and whether it is in beta or live. Each one is a tomte, our word for one production agent with one defined job.

An agent that is not on the list does not get credentials. That single rule is what keeps the count honest at 41 rather than “roughly forty, plus whatever marketing wired up last month”. Nokod’s 2026 survey of 200 CISOs found security teams could see only 44% of the AI agents their business users had built. The gap is almost never malice. It is that nobody made registration the price of entry.

What gets recorded on every run

Each run writes a row before it writes anything else. The row carries the agent, the trigger (schedule, human request, or another agent), the inputs it was given, the outputs it produced, the systems it wrote to, the duration, and the exit state.

Failures are not a separate log. A failed run is the same row with a different ending, which is why “5,450 runs, zero failures” is a countable claim rather than a mood. When the number is not zero, and there have been such stretches, the same view shows which agent, which step, and what the retry did.

How a task gets an owner

Output without an owner is noise. Every piece of work an agent produces lands as a task with a named human or a named agent against it, a due state, and an acceptance condition written when the job was defined.

The cockpit then tracks three things a chart cannot: what was produced, what was accepted, and what was quietly ignored. The third column is the useful one. It tells you which agents are generating work nobody wants, which is the fastest way to retire an agent that looked good in a demo. We wrote about the follow up layer that chases those tasks in ai task follow up automation.

The weekly report writes itself

Once runs and tasks share one store, the Monday report is a query, not an assembly job. Ours states what ran, what shipped, what stalled, and what a person had to step in and fix, per company, with the numbers attached.

It takes nobody’s morning. That matters more than it sounds, because the reason most operations reporting decays is that a human has to rebuild it every week from four tabs, and that human eventually has a better use for the hour.

Cockpit, dashboard, or spreadsheet

	Spreadsheet	BI dashboard over logs	Operations cockpit
Knows every agent exists	Only what someone typed	Only what emits logs	Registration is mandatory
Records who asked for a run	No	Sometimes	Every run
Answers an audit question	Rebuild it by hand	Partly, if retention allows	Direct query
Shows work nobody accepted	No	No	Yes, as a state
Upkeep	Weekly human effort	Dashboard maintenance	Written once, runs with the fleet

A BI layer on top of application logs gets you most of the monitoring and none of the ownership. That is the version most teams build first, including us.

What we got wrong

Version one measured activity. Runs completed, tasks created, minutes saved. All of it went up and none of it told us whether the business was better off, because an agent producing accepted work and an agent producing ignored work looked identical.

Adding the acceptance state was a small schema change and the single most useful thing in the system. It also cost us two agents, which we switched off in June 2026 after the cockpit showed nothing downstream of them was ever used.

The second mistake was alerting on everything. Real time notification of a retry that succeeded is not information. Now the cockpit interrupts a person for a failed run, a stalled owner, or an agent acting outside its declared systems, and nothing else.

What this costs to build

A cockpit of this shape is a scoped build: the registry, the run store, task ownership, and the weekly report. That is an AI Workforce Sprint, from $9,500, delivered in 4 to 6 weeks, and you own the code, the database and the accounts at handover. If you are unsure how many agents and scripts you already have running, the AI Operations Audit at $1,900 counts them first.

Running it afterwards is the part people underestimate. Agents get added, systems change, and the registry rots the moment nobody is responsible for it. Our fractional AI officer engagement covers that at $2,900 a month, cancel monthly, with one build shipped each month rather than a retainer that buys meetings.

You can see the fleet this cockpit watches, with the run and failure counts, on our agent fleet proof page. The longer write up of what those runs taught us is in 5,450 agent runs with zero failures.

If you want to know what your own agent count actually is, send us the list of automations and AI tools your teams are running and who owns each one. Start async at /start. We reply in writing within one business day, and there is no call to book.