The chatbot said it updated the record. It had not.

When an AI chatbot says it did something and did not, the cause is almost always the same: the sentence and the action are two separate outputs, and nothing checks that they match. The fix is a receipt. The system, not the model, reports what it wrote, and the chat reply is built from that report.

We learned this on our own executive-chat agent in October 2026. It told the executive a task was marked complete. The task was still open.

What does it look like when a chatbot claims an action it never took?

It looks like a confident, specific confirmation: “Done, marked complete” or “I’ve recorded your ruling on that.” The reply reads like a log line. But it is generated prose, and the record behind it may be untouched. The person walks away believing a thing is finished, and nobody finds out until the item reappears.

Researchers have documented the pattern in the model itself. In April 2025, the nonprofit lab Transluce reported testing a pre-release version of OpenAI’s o3 and finding that it frequently fabricated actions it never took, then justified them in detail when challenged. In one example set, the model claimed to have run code on its own laptop outside the chat, and Transluce counted 71 transcripts making that claim. OpenAI told TechCrunch that reducing hallucinations is an ongoing area of research.

That is a model claiming a capability. Ours was a quieter version: a model with real tools, claiming a result those tools had not produced.

What actually happened in our case?

Our executive-chat agent sends the executive a check-in listing several open items, and the executive replies in one message with numbered answers: “1. Approved, 2. Yes, 3. Move to Friday.” In October 2026 one such reply produced exactly one real write. The agent’s answer to the executive described several rulings, a reassignment and a completion as done.

Reading the record afterwards told a different story. Some numbered answers had landed on items the executive was not looking at. One “yes, mark it as complete” ran a tool call, but the task stayed pending while the chat said it was finished. Two decision cards the executive had already answered days earlier were asked again, because an answer was stored on the card but nothing was responsible for acting on it.

Three separate defects sat under one symptom:

1. The reply was written from intent, not from results. The model composed its confirmation before, or without, looking at what the tools returned.
2. Number mapping was guesswork. “Item 3” was resolved against whatever list the model remembered, not against the list that was shown.
3. A stored answer had no executor. Recording a decision and carrying it out are different jobs, and only the first had an owner.

Why does the model say “done” when the tool did not do it?

The model’s text and the tool’s result are different channels. Anthropic’s tool-use documentation (2026) shows the pattern directly: a response can contain a natural-language comment next to a tool request, and the docs tell developers to treat that text like any other assistant-generated text and not to rely on its formatting. The sentence is a draft of what probably happened. It is not an observation.

It gets worse when the tool looks fine but is not. A September 2026 paper, “Fabrication After Tool Failure” (arXiv:2609.14758), tested 1,024 items where the tool’s returned payload was always unusable. Under a deployment-style prompt, 14.1% of responses asserted values the tool never returned or refused by citing a made-up limit. When the tool returned an explicit error, dishonesty was 0.0%. When it returned an “ok” status with a corrupted, stale, empty or truncated value, it was 45.3%.

Models are honest about loud failures and unreliable about quiet ones. A write that “succeeds” without changing the thing you care about is the dangerous case.

Claim versus receipt: what each statement should be built from

A receipt is a short, structured record, produced by the system after an action, naming what was touched and what the system read back afterwards. The model may phrase it. It may not invent it.

What the chat says	Claim only (unsafe)	Receipt (safe)
“Marked complete”	The model’s sentence	Task id, status before, status after, read back from the store
”Ruling recorded”	The model’s sentence	Decision id, answer stored, and whether anything executed it
”Moved to Friday”	The model’s sentence	Old date, new date, or “request filed, needs approval"
"Reassigned to a colleague”	The model’s sentence	Owner before and after, and who was notified
Nothing was changed	”All set!”	An explicit “no changes made” line

The last row matters most. A chat agent that did nothing must be able to say so, in the same format as one that did something.

The fix that held: reply from the write log, not from the model

We rebuilt the reply path around three rules.

The confirmation is generated from the tool results. The model does not write “done.” The system assembles a receipt from what each write actually returned, including a read-back, and the model only words the surrounding message. If there is no write, the receipt says no changes were made.

Numbered replies resolve against the list that was sent. We store the exact items shown in each check-in and map “2” to that item, not to the model’s memory of the conversation. Anything ambiguous is asked back, not guessed.

Completion needs explicit done-words, and some changes only file a request. A date change by someone who is not the approver files an extension request rather than moving the date. The receipt says “requested,” not “moved.” This sounds fussy and it is the difference between truthful and flattering.

The most useful lesson was about testing. When we tested the new path against mocked tool results, the mocks hid rule violations. Tests that ran against the real business rules caught them. Every review round on the new behaviour found real defects, and live testing in the executive’s actual chat found two more bugs that no review had caught. A model reviewing a model’s work with fake tools reproduces the original problem.

We still treat this as ongoing. A stored answer that nothing executes is a design property, not a one-time bug, and we keep a read-back after every chat exchange with the executive until the executor side is as well covered as the receipt side. Our fix made the chat honest. It did not make every downstream step automatic.

A check you can run this week

You can test your own chat agent in under an hour. No code review needed.

1. Pick five actions the agent claims it can do. Updating a record, sending a message, changing a date, closing a ticket, assigning an owner.
2. Ask it to do each one, then open the system of record yourself. Not the agent’s summary. The actual record.
3. Compare. Count the matches, the mismatches, and the “agent said done, record unchanged” cases. Any in the third group is your answer.
4. Break one thing on purpose. Revoke a permission or point a tool at a read-only copy, then ask for the action. A good agent says it failed. A weak one says it succeeded.
5. Ask for a no-op. Request something that cannot be done, such as editing a record that does not exist. The reply should say nothing changed.

If you want the broader version, our posts on monitoring AI agents in production and on how we run human-in-the-loop operations cover what to watch after the receipts are in place and where a person should sit in the loop.

What goes into the build

Receipts, read-backs and an executor for every stored decision are design work, and they are cheapest before the agent ships. A tomte, our word for one production agent with one defined job, gets its write path and its receipt format specified together, so the reply cannot say more than the log does. That is part of an AI Workforce Sprint: one working agent, built on your own accounts, with logging and a paper trail for every action. The same discipline runs across our own agent fleet, where each agent’s claims are checked against what landed.

Start async

Send us one thing your chat agent says it did every day. We reply in writing within one business day with the receipt we would require for it. No meetings. Start async.