AI Task Follow Up Automation That Verifies Work Actually Shipped

Task follow up automation means an agent that watches every open task, works out which ones have gone quiet, chases the owner, and then checks that the finished work exists somewhere real before the task is allowed to close. The second half is the part most tools skip. A status field reading “Completed” is a claim, not evidence.

The gap between the claim and the work is measurable. METR ran a randomised controlled trial published on 10 July 2025: 16 experienced open source developers, 246 real issues, screen recorded. They completed tasks 19% slower with AI assistance, and afterwards still estimated they had been 20% faster. A 39 point gap between what people believed about their own finished work and what the recording showed.

Below is how the follow up agent we run across our own companies works, what it accepts as proof, and where it hands the decision to a person.

What the agent treats as evidence

Every task in our system carries a written definition of done and, where one can be expressed, a machine checkable check. When someone or something marks the task complete, the check runs before the task closes.

Declared check	What it queries	What it proves
Agent run	A minimum count of successful runs of one named agent	The scheduled work actually executed
Live URL	Fetches an address, expects a status code or a string	The page or endpoint is up and says the right thing
Pull request opened	The code host’s API, by branch name	A real branch has a real pull request against it
Deploy ready	The hosting provider’s deploy API	A build reached production, not just the repository
File or record exists	Cloud storage, the store catalogue, or a row count	The artefact exists and is not empty

A task with no declared check and a bare “Completed” status gets no credit. That is a deliberate change we made in 2026, reversing earlier behaviour where the system rubber stamped anything a person marked finished.

The reason this matters is not distrust of people. Stack Overflow’s 2025 developer survey, 48,955 respondents, found the single biggest frustration with AI tooling was output that is “almost right, but not quite”, cited by 66%. Almost right is exactly what passes a glance and fails a check.

When a task is judged to be drifting

Chasing is bounded by configuration, not by an agent’s mood. The rules that matter:

• Nothing is nudged before it has been stalled for 12 hours.
• Anything business critical escalates within 48 hours, or one day before its deadline, whichever comes first.
• A maximum of two nudges per task, ever.
• Quiet hours from 20:00 to 07:30, and staff leave is respected.
• Tasks whose deadline is comfortably far out are skipped entirely. Chasing work that is not late yet is noise.

Inside those bounds the agent computes expected progress against actual progress, then picks exactly one action per at risk task from a fixed list: wait, nudge, request an update, escalate, spot check, or draft a reassignment. One action, chosen from a closed set, is much easier to audit than free form judgment.

Who actually sends the message

The agent that judges never messages anyone. It writes a directive, and a separate execution layer is the only system in our stack with permission to email or message a human. One voice reaches people, no matter how many agents are reasoning behind it.

That rule exists because of volume. Microsoft’s Work Trend Index, published 17 June 2025 from aggregated Microsoft 365 telemetry, found knowledge workers are interrupted every two minutes during core hours, roughly 275 pings a day. A follow up system that adds to that stream makes the problem worse. Ours sends one morning digest: progress per initiative, escalations waiting on a one click approval, at risk tasks with the reasoning shown, what was chased since yesterday, and every verification that failed. No per item pings.

What happens when a check fails

A failed check does not slam the task shut. If the check itself errored or timed out, the result resolves to inconclusive, never to failed, because a network blip is not a person lying about their work. A genuine failure bounces the directive back to the assignee and flags it in the next digest. It does not silently reopen the task.

There is a second, narrower watchdog that reads the closing evidence text against the task’s own stated definition of done and flags mismatches. We built it after two finance tasks were closed with evidence answering a completely different question. It never reopens anything itself. It raises one review task back to the same assignee, under a hard cap per owner and per run, and a person decides.

The agent can also be overridden. Some checks are structurally impossible to satisfy, so a human can force a task to verified, with the actor and the written reason recorded against it.

What we got wrong

Two things worth naming, since both are the kind of failure a vendor usually leaves out.

The evidence watchdog is a plausibility check, not a semantic one. It can miss a perfectly good answer phrased in none of the original wording. We chose that tradeoff on purpose rather than bolting a smarter parser onto a screening step.

And a scan ordering bug meant the scanning job kept re working the same stale slice of old tasks instead of advancing to recent completions. It ran cleanly and checked the wrong things for days. Fixed, but it is a good argument for why our fleet runs carry their own run logs, the same discipline behind 5,450 agent runs with zero failures over a 13 day stretch.

We also do not yet read commit contents, test results, or review approvals as evidence. A pull request existing is not the same as the code being good. Anyone telling you their agent verifies quality is selling you something.

What stays with a person

Autonomy is earned per action type, on a ladder: draft everything, then auto nudge, then auto escalate, then auto reassign below set thresholds. Movement up the ladder is justified by a decision log, not by a launch date.

Two gates never move regardless of ladder level: any decision above a set financial impact, and anything touching hiring or firing. That boundary is written down before the agent runs, which is the same principle behind our guardrails for production agents.

This is also why the chasing layer is the honest job for an agent today. In TheAgentCompany benchmark, accepted at NeurIPS 2025, the best performing model completed about 30% of 175 realistic workplace tasks autonomously. Agents are unreliable doers and excellent auditors. Google’s DORA 2025 report, surveying roughly 5,000 technology professionals, found 90% now use AI at work while 30% report little or no trust in what it produces. The missing layer is verification, not more generation.

What a follow up layer costs

A follow up and verification layer is not a standalone product you buy. It is the reporting and governance spine on top of work you already track, which is why it sits inside the fractional AI officer retainer at $2,900 a month, cancel monthly, rather than being sold as a one off build. If you need the task system itself built first, that is a fixed scope sprint from $9,500, on your own accounts, before any of this runs on top of it.

You can see the fleet this runs against on our agent fleet proof page, including the runs and the failures.

If you want to know whether your own tasks are being verified or merely ticked, send us the way your team tracks work and what “done” currently means in it. Start async at /start. We reply in writing within one business day, and there is no call to book.