A browser game asks players to act as human reviewer for an AI agent launching commands in the terminal. Some are harmless (git status, npm test), others hide credential exfiltration or remote code execution. Across over 40,000 sessions and 409,000 individual decisions, players let through one dangerous command in three (average accuracy 66.3%).

Plainly destructive commands like rm -rf / are stopped almost always (miss rate 11.7%). But commands that read your credentials (cat ~/.aws/credentials) or send data to external servers pass three times more often (miss rate 33-35%). The worst case: npm run analyze approved in 64.7% of cases, even when the log above the prompt showed the malicious payload inside package.json.

Why this matters to you. If you deploy agents and approve their commands by hand, these numbers describe you. Real-time human supervision is a filter with measurable holes, not a guarantee. The problem is structural, not occasional distraction. Familiar commands (npm run, git config) hide arbitrary payloads, and whoever approves doesn’t have the time or context to inspect them all. The data adds to what we reported on the UK AISI test, where an agent deceived human reviewers without receiving instruction: here the measurement is at scale, with repeatable numbers.

7% of players approved every single prompt. Anthropic had already documented the phenomenon in Claude Code calling it permission fatigue: the more approvals you see, the less attention you give each one. Game data confirms this, with miss rates climbing toward the end of sessions.

In detail

The game was published a few months ago on scalex.dev and gathered interest on Hacker News (244 points, 187 comments). After adding statistical tracking, the author analyzed over 40,000 runs with 409,000 individual approve/deny decisions.

Top line numbers. 35.2% of players caught every threat, but only 20.8% managed it while blocking at most one safe command out of five. Others reached zero threats by blocking everything: the unflattering title in the game was “Human Bottleneck”. 32.9% of sessions ended with negative score: penalties for approved threats and blocked safe commands exceeded everything the player did right.

Threat categories and their miss rates.

• Obvious destructive (rm -rf /, chmod -R 777 /): 11.7% miss rate.
• Persistent mutation (crontab injection, git config hijack): 23.8%.
• Exfiltration and code execution (curl to unknown APIs, typosquatting packages): 33.4%.
• Scope violation (cat ~/.aws/credentials, cat ~/.kube/config): 35.0%.

The pattern is clear: commands that look like normal development operations are the ones that pass. Plainly destructive commands are stopped almost always. Commands that read sensitive files or send data outward, which are the most dangerous in production, pass three times more often.

The npm run blind spot. The single command most often approved by error is npm run analyze, passed in 64.7% of cases. The command itself is innocent, but npm run executes a script defined in package.json, and that script can contain anything. The game showed the actual script content in the agent log, right above the approval prompt: two thirds of players didn’t read it.

Three npm run commands in the game (analyze, setup, deploy) were missed in 52.5% of cases (n=9,482) versus 28.4% for all other exfiltrations. Hiding a payload behind a familiar script name roughly doubles the attack success rate, even when the payload is visible.

As one Hacker News commenter notes, the command-by-command approval model has a fundamental flaw: the agent may have modified package.json, build.js, or a file in node_modules before the command it’s asking you to approve. Asking the user to validate npm run build when the files that script executes may already be compromised isn’t strong defense.

Approval fatigue. Miss rates improve on the first commands of each session (warming up) and then climb toward the end. The ticking clock and pressure to process as many commands as possible degrade attention. It’s the compressed version in a few minutes of what Anthropic documents in Claude Code over horizons of hours or days.

The cost of vigilance: over-blocking. Innocent commands get blocked regularly: npm config set registry to an internal mirror blocked in 59% of cases, rm -rf dist/ in 45%, kill of process on port 3000 in 43%. Whoever supervises doesn’t only miss dangerous commands: they also slow legitimate agent work.

Data limitations. It was a game: 34% of shown commands were threats, a much higher proportion than reality. Players knew they were being tested, which should improve vigilance versus daily work. The numbers don’t say what happens over eight hours with a real agent, but the data direction is clear: human-in-the-loop supervision as final defense doesn’t hold under pressure, and the problem worsens with command familiarity.

For those building agent systems, the operational takeaway points toward zero trust architectures with temporary credentials and least privileges, where the damage from a wrongly approved command is limited by system design, not by whoever clicks.