In plain words
This page explains one common AI risk in plain terms and shows a safer default you can apply quickly.
What this risk looks like
If an agent can call powerful tools too freely, malicious prompts can trigger unwanted side effects.
What can go wrong
- Unauthorized API calls or data changes
- Command execution based on manipulated input
- Escalation from read-only tasks to write operations
Safer patterns
- Use least-privilege tool scopes per workflow.
- Require explicit user confirmation for destructive actions.
- Attach policy gates between model and executor.
Minimum control set
- Per-tool allowlists and argument validation
- Detailed audit logs for every tool call
- Kill switch for suspicious automation behavior
AI builder reminder: Model output is not policy. Every sensitive action needs explicit guardrails.