AI agent security: a threat model for autonomous agents

Cross-post. Original: stellarbytecapital.com/blog/ai-agent-security-threat-model While a chatbot only produces text, an autonomous agent takes actions: it calls tools, runs code, moves data, and spends money. That shift changes the security problem entirely. "Is the prompt safe?" is no longer the question. The question is: what can this agent do, and what stops it when it goes wrong? Treating agent security as prompt filtering is how teams end up with an impressive demo and a production incident. You need a threat model. Here's ours. An agent's exposure lives on three distinct planes. Confusing them is why defenses miss: The model plane — what goes into and out of the LLM: prompts, retrieved documents, tool outputs fed back as context. The action plane — the tools the agent can invoke: shell, HTTP, database, file I/O, payments, and third-party tools / MCP servers. The runtime plane — where the agent's code and tools actually execute: the process, container, host, and network. Prompt filtering only touches the first plane. Most real damage happens on the second and third. Threat Vector Primary control Prompt injection Malicious instructions hidden in a page/doc/tool output the agent reads Treat retrieved content as untrusted; confirm before consequential actions Tool misuse / over-permission Broad tools (shell, DB write, payments) the agent is steered into abusing Least privilege; gate high-impact tools behind explicit approval Code-execution escape Agent runs generated code that breaks out of its sandbox Disposable one-shot containers; gVisor/Firecracker for multi-tenant Data exfiltration Agent sends private data to an external endpoint Default-deny egress; allowlist required hosts; log every call Credential theft Agent code reads API keys mounted into its environment Keep secrets out of the runtime; proxy authed calls through a trusted layer Supply-chain / malicious tools A third-party tool or MCP server behaves adversarially Vet and pin tools; isolate them like generated code Unbounded cost Runaway loop burns tokens, CPU, or spend Per-user quotas; hard caps on iterations, tokens, wall-clock Cross-tenant leakage One user's run leaks state into another's No reuse between runs; isolate state per execution Least privilege everywhere. Every tool, mount, and network path defaults to off. Assume the code and the content are hostile. Generated code, retrieved docs, and tool outputs are all untrusted input. Human-in-the-loop for consequential actions. Reading is cheap; sending money or deleting data should require a confirmation gate, not the model's judgment alone. Isolate execution, deny egress. An escaped or hijacked agent with nowhere to send data is a contained one. Audit everything. You cannot secure what you cannot reconstruct. Prompt filtering asks "will the model say something bad?" A threat model asks "when something goes wrong, what's the blast radius?" — and shrinks it to near zero. We're Xingyao Byte — building secure AI-execution layers, quant trading systems, and payment platforms. Remote, async-first → stellarbytecapital.com