Agent security is a platform problem, not a prompt engineering challenge

Autonomous AI agents often exceed their intended scope not because of malicious intent, but due to the gradual accumulation of tool access and credentials. In a recent analysis, CNCF ambassadors William Rizzo and Francesco Beltramini argue that this boundary erosion is a structural infrastructure issue rather than a failure of agent logic or governance processes.

What happened

The core issue identified is that an agent’s effective power is defined by the tools it can access, which often expands silently over time. A credential scoped for a demo might survive into production, or a delegation added to unblock a specific deploy becomes a permanent permission. This drift occurs because authority is granted incrementally and locally, without a central decision point. By the time teams realize an agent can perform an unauthorized action, it has likely had that capability for weeks.

Many organizations attempt to solve this through governance boards or stricter review processes. However, these methods govern human decisions, whereas the problem stems from capabilities that arrive without any explicit decision. The authors contend that process alone cannot review a class of change that never surfaces as a distinct event. Instead of relying on vigilance, systems need a substrate that enforces boundaries automatically, regardless of whether anyone is watching.

How it works

Relying on the agent to police its own limits via instructions or prompts is structurally flawed. An agent’s output is a probability distribution, meaning it will usually choose the correct action but may fail under novel or adversarial conditions. More critically, even a perfectly compliant agent will execute any action its tools permit. If an agent holds a tool that moves money, it can move money, regardless of instructions saying otherwise. Capability always overrides intent because capability is what actually executes.

To address this, the authors propose a Safe Operating Model based on concentric rings of least privilege, treating the agent as a standard workload. The center is Identity, using short-lived, attested credentials so every downstream check knows exactly what is acting. The first ring, Reach, uses a service mesh to enforce which services an agent can connect to, preventing silent graph growth. The second ring, Capability, separates admission (what tools an agent can hold) from execution (what it can do with them), requiring per-call authorization.

The keystone of this model is ensuring the agent cannot modify its own configuration. An agent must be denied the ability to edit the policies or objects that define its authority. Changes to boundaries must come from version-controlled configuration managed by humans or pipelines. This creates a deterministic spine for regulated workflows, where the agent provides recommendations or drafts, but the platform enforces the final gates and approvals.

Key details

• Agent risk stems from accumulated tool access rather than intentional design flaws.
• Prompt-based constraints are probabilistic suggestions, not guaranteed security boundaries.
• Identity must be verified via short-lived, attested credentials at the platform level.
• Reach is enforced by service meshes, blocking connections to undeclared peers.
• Capability is split into admission control and per-call authorization at the tool gateway.
• Agents must never have permission to modify the configuration objects that define their own limits.

Why it matters

For software engineers and technical leads, this shifts the security burden from the application layer to the infrastructure layer. It means that building agentic systems requires integrating cloud-native security primitives like workload identity, service meshes, and policy-as-code. Teams can no longer treat agents as black boxes that manage their own safety; they must explicitly declare and enforce the boundaries within which the agent operates.

This approach also changes how auditable and regulated workflows are designed. By separating the agent’s cognitive work from the deterministic execution of outcomes, organizations can maintain compliance. The platform records evidence of identity, reach, and capability checks as a byproduct of operation, providing a clear trail for regulators. This allows agents to handle complex interpretation tasks while ensuring that critical actions like fund transfers or data modifications pass through immutable, reviewed gates.

What you can do

• Treat agent identities as first-class workload identities with short-lived, attested credentials.
• Implement service mesh policies to restrict agent connectivity to only declared peers.
• Use admission controllers to prevent agents from binding to unapproved tools.
• Enforce per-call authorization at the tool gateway to validate each specific action.
• Ensure agent identities are explicitly denied permission to modify their own policy configurations.
• Design workflows where agents emit proposals or drafts, leaving final execution to deterministic pipelines.