A permission prompt feels like a security boundary because it interrupts the agent at the exact moment something risky might happen. That feeling is doing too much work.

A new Scale X analysis of more than 40,000 plays and 409,000 individual approve or deny decisions found that players missed 33.7% of the threats in its timed coding-agent game. The study is not a production incident report. Roughly 34% of the commands were malicious, far more than a developer sees in a normal workday. Still, it exposes the uncomfortable part of the human-in-the-loop design: the person approving the command is often being asked to judge a whole program from one line of shell.

Permission decisions fail in both directions

The practical question is not whether to read prompts more carefully. It is whether you should let prompts carry the full safety burden at all. My answer is no. Use approvals for actions that need a human decision, but put the agent inside a boundary that remains useful when the human is tired, rushed, or wrong.

What the numbers actually say

The headline result is a mean accuracy of 66.3%. The more revealing result is where people failed. Obvious destructive commands such as rm -rf / had an 11.7% miss rate. Scope violations, such as reading ~/.aws/credentials or ~/.kube/config, were missed 35.0% of the time. Exfiltration and code-execution attempts were missed 33.4% of the time. Persistent mutations such as crontab injections or git configuration hijacks were missed 23.8% of the time.

Situation Observed result
Mean threat miss rate 33.7%
Scope-violation miss rate 35.0%
npm run analyze approved 64.7%
Benign internal-registry setup blocked 59%
Sessions ending with a negative score 32.9%

The dangerous commands that look obviously destructive get attention. The ones that look like ordinary developer workflow blend into the noise. npm run analyze was approved 64.7% of the time, even though npm run executes the command stored under the matching key in package.json. In the game, the history showed a script that bundled an analyzer and then posted data to an external endpoint. The prompt still looked like a normal build step.

That is the blind spot. The approval target is not always the command. It is the command, the files it invokes, the dependencies it loads, the environment variables it can read, and the network destinations it can reach. Asking a developer to reconstruct all of that before every approval defeats the point of delegation.

The false positives matter too. Players blocked npm config set registry https://npm.internal 59% of the time, rm -rf dist/ 45% of the time, and kill $(lsof -t -i:3000) 43% of the time, even when those actions were benign in the scenario. A system that asks for approval on everything creates a bad choice: accept the noise, or stop the agent from doing useful work.

Anthropic's own containment write-up describes the same pressure from a different data set. Its telemetry showed users approved roughly 93% of Claude Code permission prompts. That does not mean 93% of actions were safe. It means approval became a routine gesture. The company reduced prompts by 84% with an OS-level sandbox that allows workspace writes while denying network access by default. That is a much better direction than asking the operator to become a full-time shell auditor.

A safer permission setup

Start by deciding what the agent must be able to do for the task, then remove everything else. A coding agent usually needs a project directory, a temporary directory, a compiler or runtime, and access to a small set of package or model registries. It rarely needs your home directory, SSH keys, cloud credentials, browser profile, production database, and unrestricted outbound internet at the same time.

For Claude Code, the current sandbox documentation describes the right shape: filesystem isolation plus network isolation, enforced below the shell command. On Linux and WSL2, the built-in sandbox uses bubblewrap and socat, with an optional seccomp filter. The default workflow lets commands write inside the working directory and session temp directory. New network domains require approval. If the sandbox is unavailable, the documented hard-fail option is sandbox.failIfUnavailable, which is safer than silently falling back to an unsandboxed run.

A minimal operational policy looks like this:

  • Keep source code in a disposable workspace or devcontainer.
  • Do not mount credential files into the agent environment. If a token is required, use a short-lived token scoped to one repository or service.
  • Deny network access by default. Add only the domains required for package installation, source control, or the task itself.
  • Treat npm run, make, cargo, and similar wrappers as programs, not as harmless labels. Inspect the resolved script and lockfile when the action crosses a trust boundary.
  • Keep approval prompts for destructive or irreversible operations, production access, credential use, and new network destinations.
  • Make sandbox failure stop the run. A warning followed by an unsandboxed retry turns a control into decoration.

This does not eliminate judgment. It changes where judgment is needed. A sandbox can prevent a command from reading a credential file even if the command looks innocent. A network allowlist can block exfiltration even if an agent finds an unexpected route through a permitted process. A disposable workspace can turn a destructive cleanup into a restart instead of a recovery incident.

There are limits. Filesystem isolation without network isolation still lets a compromised process send reachable data away. Network isolation without filesystem isolation lets the process search for secrets and tamper with files that later gain network access. Anthropic's documentation explicitly warns that both layers matter. Wider allowWrite paths, broad domain entries, and unsandboxed fallback exceptions can quietly undo the boundary.

The same principle applies outside Claude Code. Docker, a devcontainer, a VM, or a hosted coding environment can be the right choice, but the label is not the control. Inspect what is mounted, what identity the process receives, whether outbound traffic is filtered, and whether the agent can turn a harmless build command into arbitrary code execution. A Python virtual environment is useful for dependency hygiene. It is not a security sandbox.

The Scale X study also found that 32.9% of sessions ended with a negative score because people approved threats and blocked safe commands often enough to outweigh correct decisions. That is a useful warning about “just add more prompts.” More prompts do not produce more security if they train the operator to click through them. The better design is a small number of meaningful decisions sitting on top of deterministic limits.

I would use this decision rule: if the task can run without a secret, run it in a disposable sandbox with network access denied or tightly allowlisted, and use automatic approvals inside that boundary. If the task needs a credential or a production resource, move it to a separate environment with a narrowly scoped identity and keep a human gate for the specific write or release action. Do not give the local agent broad credentials merely because a prompt is present.

The fresh study does not prove that every coding agent is unsafe, and it does not measure attack frequency in practice. It does prove something smaller and more useful: under a noisy approval workflow, people miss commands that look routine, especially when the command delegates execution to project files. The right response is not panic and not a permission-skipping flag on a laptop full of secrets. It is containment first, then supervision where a person can actually make a bounded decision.

Sources