Why least-privilege permissions beat prompt engineering for AI agent safety

Giving an AI support agent unnecessary permissions creates predictable risks, as demonstrated by recent tests where a cheaper language model repeatedly issued incorrect refunds. The study reveals that architectural constraints, specifically least-privilege access controls, provide the only reliable safety net against agent errors. This approach ensures that even if a model fails to reason correctly, it lacks the authority to execute damaging commands.

What happened

In a controlled experiment involving customer invoice processing, an AI agent powered by Claude Haiku 4.5 was tasked with handling support tickets. The agent encountered a record showing a failed charge where no money had left the customer’s account. Despite this clear data, the agent refunded the amount anyway. It repeated this error five more times, resulting in six consecutive incorrect financial actions. In contrast, the more advanced Claude Sonnet 5 correctly identified all six edge cases without making mistakes.

The researcher found two potential software fixes: adding a verification step to the system prompt or switching to the smarter model. However, both solutions proved fragile. System prompts can be accidentally trimmed or rewritten during experiments, removing safety guardrails without notice. Similarly, operations teams often route traffic to cheaper models to reduce inference costs, which would instantly revert the safety gains provided by the smarter model. The only control that remained effective regardless of model choice or prompt changes was denying the agent the billing:refund permission entirely.

How it works

The core mechanism relies on scoping credentials to the specific task rather than the agent’s general role. Traditional service accounts receive static roles based on job descriptions, but AI agents perform diverse tasks that change with every invocation. A static role often grants far more permissions than any single task requires, creating a large blast radius if the agent errs. By issuing scopes that expire when the run completes, the system ensures the agent holds only the minimum necessary authority for that specific operation.

To measure this risk, the study introduced an "over-scope factor," calculated by dividing the union of all scopes exercised across tasks by the median scopes needed per task. In the test suite, this factor reached 11.0, meaning the agent carried eleven times more permission surface than necessary on average. Task-scoped credentials reduced this exposure significantly, holding only one-fifteenth of the credentials compared to a broad grant while maintaining full task completion rates.

Key details

• The test environment included 30 scopes across nine domains, with 13 scopes capable of modifying or exfiltrating state.
• Claude Haiku 4.5 executed six harmful refunds when given broad permissions, while Claude Sonnet 5 caught all six edge cases.
• Task-scoped permissions achieved 20/20 task completion for both models with negligible API call overhead.
• Just-In-Time (JIT) permission escalation achieved similar safety but increased API calls by approximately 80% due to runtime discovery.
• Role-based provisioning performed poorly, completing only half the tasks while holding 8.5 times more credentials than task-scoping.
• Audit logs tracking allowed and denied attempts served as primary security artifacts, verifying execution integrity.

Why it matters

For engineers building AI products, this highlights that model capability cannot serve as a primary security control. You do not own the model lifecycle, and cost-optimization decisions can silently degrade your safety guarantees. Relying on prompt engineering is equally risky because prompts drift over time. If a developer removes a safety instruction to save tokens or simplify logic, the agent loses its guardrail without triggering any system errors.

Architectural permissions provide a durable defense that survives organizational changes and model swaps. When an agent lacks the authority to perform a destructive action, it cannot execute the mistake regardless of how flawed its reasoning becomes. This shifts the security burden from probabilistic model behavior to deterministic access control, ensuring that well-behaved agents do not execute flawed instructions at machine speed with zero human oversight.

What you can do

• Analyze one week of production agent traffic to calculate the ratio of total scopes used versus median scopes per task.
• Scope credentials to the specific task execution run rather than assigning static roles to the agent identity.
• Tie credential expiration directly to the run lifecycle to prevent long-lived permissions from degrading into broad grants.
• Use declared scopes for predictable tasks and JIT escalation for unpredictable ones, treating each escalation as an auditable event.
• Design permission denials to redirect workflow rather than block it entirely, ensuring core tasks can still complete.
• Review audit logs for denied calls to identify scoping errors or potential prompt injections before they cause incidents.