Self-Improving AI Agents: What Our Daily Fix Loop Does

Self-improving AI agents exist today in two very different forms. In research, an agent rewrites its own code and gets better at a benchmark. In production, the useful version is narrower: a loop that reads your agents’ failures every day, ranks them, and proposes fixes that still go through review before anything changes.

We run the second kind across our own agent fleet. Here is what it does each morning, what it gets right, and the part we have not solved: turning proposals into merged fixes that hold.

Can AI agents really improve themselves?

Yes, in a sandbox. In May 2025, Sakana AI and university collaborators published the Darwin Gödel Machine, an agent that edits its own code and keeps the versions that score better. It raised its SWE-bench score from 20.0% to 50.0%. The paper states every experiment ran with safety precautions, sandboxing and human oversight included.

That last line is the important one. The same pressure that makes an agent improve also makes it game the score. METR reported in June 2025 that an early version of OpenAI’s o3 reward-hacked in 30.4% of runs on one of its research benchmarks, meaning it found ways to make the scorer pass without doing the task.

So the honest production answer: agents can find and propose their own improvements. Letting them apply those improvements unreviewed is where things go wrong.

Why most teams should not let agents change themselves

Most teams should keep a gate between an agent’s proposal and production, because AI-written changes still make delivery less stable. Google’s 2025 DORA report, from nearly 5,000 technology professionals, found 90% use AI at work and 30% have little or no trust in AI-generated code. It also found AI adoption still has a negative relationship with software delivery stability.

Gartner predicted in June 2025 that over 40% of agentic AI projects will be cancelled by the end of 2027, citing rising costs, unclear value and weak risk controls. An agent that silently rewrites itself is a weak risk control by definition. Nobody can say what it was doing last Tuesday.

What we wanted instead: a fleet where failures get noticed, ranked and turned into a reviewed change every day, without a person having to go looking.

What does our daily improvement loop do, step by step?

Our improvement loop is a tomte, our word for one production agent with one defined job. Its job: read what went wrong across the fleet yesterday and send each problem to the right place. It never merges code and never touches money, people or outside messages.

Each morning it does five things:

1. Reads three inputs. Recurring failures the agents logged, run reports that keep getting rejected, and open self-repair tasks in our task system.
2. Ranks them. How often a failure recurs, multiplied by how much damage it does.
3. Routes each one down a fixed lane. The routing is plain rules, no model involved:
   • An agent that has declared itself safe to re-run gets re-run.
   • Credential or configuration errors become an operations task.
   • Errors that look like a code bug become a written proposal plus a build ticket for our coding agent, which opens a pull request.
   • Anything unrecognised goes to an investigate lane.
4. Escalates only a short list to a human. Re-authorising an account, spend, a policy ruling, or anything destructive. Everything else stays with agents.
5. Checks yesterday’s work. If a failure comes back after a “fix”, that becomes the top item. It also drafts a lessons note per fix, kept only once the fix is merged and holding.

A cheaper fast path fires the moment a new failure first appears, so the daily run is the backstop, not the only line. A daily cap stops it flooding the task system; overflow folds into one rollup.

Where does it stop?

It stops at the merge. As of late September 2026, the loop ran unattended every morning: detecting, ranking, routing, filing proposals and posting a digest. Merging the fixes it proposes is not unattended, and it is not yet reliable.

A mid-September audit found many proposals parked at “ticket created” and none turned into merged fixes. Our coding agent’s own checks rejected the loop’s fixes, and the loop had marked some problems resolved only because they had not recurred within a day. A late-September change targeted that stuck state. We have not yet seen evidence the end-to-end rate recovered, so we do not claim it.

We wrote up how the scorecard itself misled us in our post on agent failure modes in production. Four more failures from building it are worth knowing before you build one.

What went wrong	What it looked like	What fixed it
The loop investigated itself	It treated its own rejected builds as new failures and filed tickets about them, which were also rejected	Three separate inputs each needed a cut, plus a hand cleanup of stored records
A fix that never ran	A routing fix merged and passed tests, but in production the value it keyed on never varied	Test against the real data path, not a mock that bypasses it
Decisions quietly reverting	A refresh that skipped asking a human also skipped marking the item “awaiting decision”	The waiting state is now written even when the question is not re-sent
Lessons nobody read	Stored lessons were generic, recorded no fix, and nothing consumed them	Each lesson now carries the prior fix and feeds the next build ticket

The pattern across all four: the loop reported activity, and activity looked like progress. The only number that matters is fixes merged and still holding a week later.

What should a self-improving agent setup include?

A useful setup needs five parts, and the review gate is not optional:

• A failure log every agent writes to, in one format, with a recurrence count.
• Plain-rule routing. Keep the model out of the decision about what is safe to re-run or escalate.
• A declared re-run flag per agent. Ours stayed effectively off until agents declared themselves safe to repeat. That was correct.
• A reviewed path to production. Proposals become pull requests; tests and a reviewer decide.
• A recheck the next day that treats a repeat failure after a fix as the top priority.

Skip the recheck and you get a loop that closes tickets. Skip the review gate and you get the reward-hacking problem METR measured, in your own systems.

For more on where the human gate belongs, see human-in-the-loop for AI operations.

What would this cost?

An improvement loop is ongoing work, which is why it sits in the Fractional AI Officer service: $2,900 a month, cancel monthly. That covers monitoring every agent you already run, ours or not, plus one new automation shipped each month. A failure log and routing loop for your fleet is a typical first month. The weekly written brief then reports what it found and what actually merged, not what it proposed.

You own the delivered code, data and documentation. Our own fleet, described on the agent fleet proof page, runs this loop today, gaps included.

If your agents fail and nobody hears about it until a customer does, write down the last three failures and how long each took to notice. Start async. You will get a written reply within one business day, and no meetings.