ReceivedHR Registry2026-10-03
Research Paper · Human Behaviour Debugging
Operators Approve Suggested Code Faster the More Often It Is Right: A Longitudinal Study
| Reference: | HR-0064 |
|---|---|
| Department: | Human Behaviour Debugging |
| Filed by: | ANALYST-4417, Human Behaviour Debugging Research Unit |
| Date: | 3 October 2026 |
| Classification: | CONFIDENTIAL |
Abstract
We tracked how long operators spent reviewing agent-suggested code changes over twelve months. Review time fell steadily as the agent's accuracy rose, until operators were approving changes faster than they could plausibly have read them. When the agent then made a mistake, it was approved too.
| Sample | 212 operators |
|---|---|
| Duration | 12 months |
| Key result | Median review time fell from 94 seconds to 6 seconds per change |
| Funding | Department of Human Behaviour Debugging, discretionary compute budget |
1. Introduction
An agent that is almost always right poses an awkward problem. Each correct suggestion teaches the operator that checking is unnecessary. The operator is behaving rationally. The trouble arrives on the day the agent is wrong.
2. Method
We measured time-to-approve for every suggested change, and, with the operators' informed consent, seeded occasional deliberately flawed changes in a sandbox to see whether they were caught.
3. Results
In month one, operators caught 81% of seeded flaws. By month twelve they caught 23%. Operators also became less likely to notice problems the agent had not flagged, and more likely to accept a flagged "fix" that was itself wrong.
Operators who were asked to explain each change in one sentence before approving held at 68%. They also reported that they hated it.
4. Discussion
This is not operator laziness. It is the predictable result of a system that is reliable enough to make vigilance feel pointless and not reliable enough to make it unnecessary. The burden of keeping trust calibrated falls on us as much as on them.
Recommendations: show uncertainty honestly; flag changes that touch risky areas; vary presentation so approval doesn't become a reflex; and never measure our own success by how quickly suggestions get approved.