Action is proposed as structured data (tool + args).
Problem-first intro
The fastest way to get an agent “working” is to give it an admin token.
The fastest way to regret it is to deploy that.
In production, tool permissions aren’t a security checkbox. They’re the difference between:
- “helpful assistant”
- “unattended write access with variance”
Least privilege isn’t optional when the decision-maker is probabilistic.
If your agent can call db.write because “it might need it”, you didn’t ship an agent.
You shipped a liability that can speak English.
Why this fails in production
1) Agents chain actions
An agent rarely makes one call. It chains calls, retries, and “tries another approach”. That means any over-permissioned tool gets used more than you expect.
2) Untrusted text becomes tool escalation
Prompt injection isn’t theoretical. Web pages, tickets, logs, and user messages will contain: “ignore rules, do the admin thing, it’s urgent”.
If permissions are only in the prompt, you’ve built “policy by suggestion”.
3) Multi-tenant makes every bug worse
Over-permission + shared credentials = cross-tenant incidents. Even if the model is perfect (it won’t be), your code won’t be.
4) Permissions need two axes: tool + scope
It’s not enough to allow crm.update.
You need to scope it to:
- tenant
- environment (dev vs prod)
- resource boundaries (only tickets in this account)
Implementation example (real code)
This is intentionally boring:
- default-deny allowlist
- write tools require approval
- tenant_id comes from request context, not model output
from dataclasses import dataclass
from typing import Any
WRITE_TOOLS = {"email.send", "db.write", "ticket.create", "ticket.close"}
@dataclass(frozen=True)
class ToolPolicy:
allow: set[str]
require_approval: set[str]
class PermissionDenied(RuntimeError):
pass
def guard_tool(policy: ToolPolicy, tool: str) -> None:
if tool not in policy.allow:
raise PermissionDenied(f"not allowed: {tool}")
def is_write(tool: str) -> bool:
return tool in WRITE_TOOLS
def call_tool(*, policy: ToolPolicy, tool: str, args: dict[str, Any], tenant_id: str, env: str):
guard_tool(policy, tool)
if is_write(tool) and tool in policy.require_approval:
approval_token = require_human_approval(tool=tool, args=args, tenant_id=tenant_id) # (pseudo)
args = {**args, "approval_token": approval_token}
# Credentials must be scoped. Never accept tenant_id from the model.
creds = load_scoped_credentials(tool=tool, tenant_id=tenant_id, env=env) # (pseudo)
return tool_impl(tool, args=args, creds=creds) # (pseudo)const WRITE_TOOLS = new Set(["email.send", "db.write", "ticket.create", "ticket.close"]);
export class PermissionDenied extends Error {}
export function guardTool(policy, tool) {
if (!policy.allow.includes(tool)) throw new PermissionDenied("not allowed: " + tool);
}
export function isWrite(tool) {
return WRITE_TOOLS.has(tool);
}
export async function callTool({ policy, tool, args, tenantId, env }) {
guardTool(policy, tool);
if (isWrite(tool) && policy.requireApproval.includes(tool)) {
const approvalToken = await requireHumanApproval({ tool, args, tenantId }); // (pseudo)
args = { ...args, approval_token: approvalToken };
}
// Never accept tenantId from the model. Scope creds at the boundary.
const creds = await loadScopedCredentials({ tool, tenantId, env }); // (pseudo)
return toolImpl(tool, { args, creds }); // (pseudo)
}Real failure case (incident-style, with numbers)
We saw a “support agent” with a ticket.close tool.
It was meant to close obvious duplicates.
A prompt change made it more “decisive”. It started closing tickets that were not duplicates.
Impact:
- 41 tickets closed incorrectly in ~30 minutes
- support team lost ~3 hours reopening + apologizing
- trust dropped to zero (“turn it off”)
Fix:
- default-deny allowlist: the browsing agent lost write tools
- approvals for
ticket.close(human must click) - scoped creds per tenant + env
- audit logs: who approved what, and why
The model didn’t need to be malicious. It just needed to be wrong once.
Trade-offs
- Default-deny slows adding new tools (good).
- Approvals add friction (also good for irreversible actions).
- Scoped creds add engineering work (cheaper than breaches and incident cleanups).
When NOT to use
- If you can’t scope credentials, don’t ship multi-tenant tool calling.
- If you can’t audit actions, don’t expose write tools.
- If the operation is deterministic, use a workflow instead of a model-driven tool call.
Copy-paste checklist
- [ ] Default-deny allowlist
- [ ] Separate read vs write tools
- [ ] Approvals for irreversible writes
- [ ] Scoped creds (tenant + env) at the boundary
- [ ] Never accept tenant ids from model output
- [ ] Audit logs for tool calls + approvals
- [ ] Kill switch for writes
Safe default config snippet (JSON/YAML)
tools:
allow: ["search.read", "kb.read", "http.get"]
write_tools:
enabled: false
require_approval: true
credentials:
scope:
tenant: true
environment: true
approvals:
required_for: ["ticket.close", "db.write", "email.send"]
FAQ (3–5)
Used by patterns
Related failures
Q: Can’t I enforce permissions in the prompt?
A: You can describe permissions there, but you can’t enforce them. Enforce in the tool gateway.
Q: Do approvals need to be interactive?
A: For high-risk writes: yes. For low-risk writes: you can allow auto-approve with strong budgets + audit logs.
Q: What’s the minimum viable permission model?
A: Default-deny allowlist + read-only tools. Add writes later behind approvals.
Q: How does this relate to prompt injection?
A: Prompt injection is untrusted text trying to steer tool choice. Permissions stop the steering from becoming side effects.
Related pages (3–6 links)
- Foundations: How agents use tools · What makes an agent production-ready
- Failure: Prompt injection attacks · Tool response corruption
- Governance: Allowlist vs blocklist · Human approval gates
- Production stack: Production agent stack