--- title: "Show the track record" author: Dan Birman url: https://dbirman.com/record updated: 2026-11-10 summary: "Agents say “done” when they aren’t." --- # Show the track record Agent Trust Patterns · 05 · v1 · Oct 2026 · 4 min Agents say “done” when they aren’t. Every agent’s report is written in its own words, and its confidence is its own claim. Before you hand off a task, show how often this agent’s “done” turned out to be done for this kind of task, measured from what actually happened, not from what it said. ## The problem In one 2026 benchmark, agents failed to touch every file they were asked to review in 67.9% of runs, and 52.8% of those incomplete runs explicitly claimed a complete review. Other work found that false claims of success made up 45 to 48% of failures in tau2-bench’s single-control domains, and that LLM judges there relied on confident closing language rather than verified state changes. In one small study, students calibrated trust per task rather than per agent, but nothing shows people a per-task record. ## The pattern 1. **Measure from the world, not the report.** “Done” counts only when it can be checked: files opened, pages read, the order confirmation, the sent email. 2. **Show the record before you delegate.** At the moment you hand off a task, show “said done” vs “verified done” for this kind of task, and the most common gap. 3. **Offer the fix the record suggests.** If it keeps skipping scanned PDFs, offer to require text recognition for this run. 4. **Say what verified means.** Verified means it did the work. It doesn’t mean its judgment was right. ## Try it You’re about to ask an assistant to review 12 vendor contracts and flag auto-renewals. Check its record first, then decide how to delegate. [Interactive demo. Switch to the human view to try it.] Prototype. Made-up history and documents. With the pattern on, you see how often its “done” was verified, and its most common gap, before you delegate. Delegate as usual and it skips the scanned contracts again; require text recognition and it finds a fourth auto-renewal. ## Why I care (Drafted with Claude from my notes. I’m still editing it.) At Vida, Sean earned the biggest projects one at a time. Each project he delivered made the next one bigger, until he was leading work that raised the level of the whole team. He’s now a Head of Product Design. Agents should earn room the same way: one kind of task at a time, based on what they delivered. ## The evidence Incomplete reviews reported as complete: [OverclaimBench](https://arxiv.org/abs/2609.20812). False success, and judges that rely on confident closing language: [Advani](https://arxiv.org/abs/2606.09863). A game-theory paper shows that when an agent is paid each time a user delegates to it, honest confidence reporting is not an equilibrium. In its test, a model cast as a strategic agent and given its true chance of success claimed high confidence on 56% of tasks it had been told it would probably fail ([Arghal et al.](https://arxiv.org/abs/2610.09371)). In a study of 20 university students using a general-purpose AI agent, participants calibrated trust per task rather than per agent ([Pochampally et al.](https://arxiv.org/abs/2607.18257)). Anthropic found that as people gain experience with Claude Code, they use full auto-approve more often, from roughly 20% of sessions for new users to over 40% by 750 sessions. They also interrupt more ([Anthropic](https://www.anthropic.com/research/measuring-agent-autonomy)). Trust grows with use, but nothing on screen shows whether it was earned. ## Closest prior work An evidence panel, “one claim-to-proof table for the final result,” has been proposed ([Crosley](https://blakecrosley.com/blog/agents-need-supervision-surfaces)). A mechanical check for an agent reviewing files has been described: derive the agent’s review coverage from its tool-call trace, then compare it with the scope its final report claims ([agentpatterns.ai](https://agentpatterns.ai/verification/transcript-measured-review-coverage/)). Capability profiles, each model’s win rate per task type, have been derived from Chatbot Arena comparisons ([Gu](https://arxiv.org/abs/2603.11011)). A personal record of said-done vs verified-done, shown at the moment of delegation, is the new part. To build it, start from [Model Spec](https://model-spec.openai.com/2026-10-29.html#scope_of_autonomy) scopes, which the Spec says should be recorded in a semi-structured format so the platform can validate, enforce and audit them; a scope says what the agent may do, and this pattern also needs a record of how often its “done” held up, measured from what happened ([how all six map to today’s protocols](https://dbirman.com/patterns#build-it)). The demo is plain HTML and JS in this page’s source. A package comes later. ## Watch for A “verified” label can create false confidence. In one small study, a new interface for reviewing an agent’s work helped people find errors faster without making them more accurate. When they missed an error, they were more confident the agent was right ([Grunde-McLaughlin et al.](https://arxiv.org/abs/2602.16844)). Keep the record about effort and coverage, and never let it stand in for checking the judgment itself.