Secure Agent Design Patterns: Dual LLM, Plan-Then-Execute

No model today reliably refuses every prompt injection, so an AI agent that reads untrusted content and holds real tools must be made safe by its architecture, not by its prompt. Secure agent design patterns are structural constraints on how an agent plans, which data reaches which model, and what tool calls are allowed once untrusted text is in play. This guide explains the six patterns published in a 2025 multi-organisation research paper, the CaMeL capability system, the dual LLM pattern and Meta's Agents Rule of Two, and shows how to choose, combine, test and measure them. It is written for architects, AppSec engineers and platform teams building agents, and goes deeper on one layer of the AI Security Architecture Reference Guide.

Key takeaways

Design for the injection that succeeds. Classifiers and guardrails lower the rate of successful injection but give no guarantee; the patterns limit what a successful injection can do.
The core idea is isolation of untrusted data from control flow. Once an agent has read untrusted text, that text must not be able to choose the next tool call.
Every pattern trades utility for security. Action-selector is close to immune but rigid; plan-then-execute keeps flexibility but still lets injected data change arguments.
Combine patterns and pair them with identity controls. Least-privilege agent credentials, egress allowlists and human approval are what make a pattern's guarantee hold in production.
Prefer narrow, application-specific agents. The authors of the 2025 design patterns paper recommend them over general-purpose agents because trust boundaries can be drawn clearly.

Why architecture beats filtering

Prompt injection (MITRE ATLAS AML.T0051 LLM Prompt Injection, and its indirect form AML.T0051.001) works because a language model processes instructions and data in the same channel. Any text the agent reads, whether an email, a web page, a code comment, a ticket or a tool result, can carry instructions. The damage comes from what follows: an agent tool call driven by those instructions (AML.T0053 AI Agent Tool Invocation) or data leaving through one (AML.T0086 Exfiltration via AI Agent Tool Invocation).

Microsoft's Security Response Center stated the design stance plainly in July 2025: its approach is not built on blocking every prompt injection, but on making sure a successful one does not lead to security impact. Defences such as spotlighting (wrapping untrusted text in randomised delimiters, datamarking it with interleaved tokens, or encoding it) and classifier-based detection like Prompt Shields reduce the success rate. Secure design patterns sit underneath them and decide the blast radius when they fail.

Probabilistic layer: system prompt hardening, spotlighting, injection classifiers, output filters. Useful, measurable, bypassable.
Deterministic layer: the design pattern, tool allowlists, typed and validated tool arguments, egress controls, scoped credentials, approval gates. These do not depend on the model behaving.
Observability layer: logging of every prompt, retrieved item, plan and tool call so that the first two layers can be tested and tuned.

Google's published approach to secure agents states the same principles in identity terms: every agent needs well-defined human controllers, its powers must be limited, and its actions and planning must be observable. See the Prompt Injection Guide for the attack side.

The six patterns at a glance

The paper "Design Patterns for Securing LLM Agents against Prompt Injections" (Beurer-Kellner et al., arXiv 2506.08837, June 2025) defines six patterns. Each one constrains the agent so that untrusted input cannot trigger consequential actions. The table summarises what each guarantees and what it costs.

Pattern	How it works	What it guarantees	What it costs
Action-selector	The LLM only maps the request to one of a fixed set of predefined actions; tool output never flows back into the model	Injected tool data cannot trigger anything outside the fixed action set	Only anticipated requests can be served; behaves like a smart menu
Plan-then-execute	The agent commits to a fixed list of tool calls before it touches untrusted data; outputs may fill arguments but cannot add or swap actions	Control-flow integrity: no new actions from injected data	Injected data can still change parameters (for example an email body); tasks whose steps depend on results are hard to plan up front
LLM map-reduce	An isolated LLM instance with no dangerous tools processes each untrusted item; a reduce step aggregates results using code or constrained formats such as booleans or numbers	An injection affects only the result for its own item	Task must decompose per item; a malicious item can still game its own score
Dual LLM	A privileged LLM plans and calls tools but never sees untrusted text; a quarantined LLM reads untrusted text but has no tools; results pass as symbolic variables	The privileged model never ingests untrusted content	Quarantined output must be constrained or kept symbolic; it can still be manipulated within its format
Code-then-execute	The LLM writes a program that calls tools and spawns unprivileged LLMs for untrusted text, then the program runs	As plan-then-execute, with the plan as explicit, inspectable code	Needs reliable code generation and a sandboxed interpreter; parameter values can still be influenced
Context-minimisation	Once the user prompt has informed an early step, it and other unnecessary content are removed from context before later steps	Injections in the user prompt cannot influence later output	Later responses cannot react to details of the original request

Dual LLM and CaMeL

Simon Willison proposed the dual LLM pattern in April 2023: a privileged LLM, a quarantined LLM and a controller written in ordinary code. The controller runs the actions the privileged model requests and stores untrusted results as variables such as $VAR1, so the privileged model only ever sees variable names. Willison himself listed the limits: it assumes user input is trustworthy, chained prompts can carry injected instructions forward, and it adds complexity.

Google DeepMind's CaMeL ("Defeating Prompt Injections by Design", Debenedetti et al., arXiv 2503.18813) develops the idea into a system. It extracts the control flow and data flow from the trusted user query, attaches capabilities to every value describing where it came from and where it may go, and enforces security policies at each tool call. On the AgentDojo benchmark the authors report 77% of tasks solved with provable security against 84% for an undefended system. The lesson for practitioners is that provenance tracking on data, checked by deterministic code at the tool boundary, is the strongest form of the pattern currently published.

The Agents Rule of Two

Meta's Agents Rule of Two (October 2025) is a simpler test that is easy to apply in design review. An agent session should combine no more than two of: [A] processing untrusted inputs, [B] access to sensitive systems or private data, and [C] the ability to change state or communicate externally. If a task needs all three, the agent should not run autonomously: it needs human approval or another reliable validation step, ideally with a fresh context window. The rule maps cleanly onto the patterns: map-reduce and the quarantined half of dual LLM remove [C] from the component that reads untrusted data; context-minimisation and data scoping shrink [B].

Choosing a pattern

Start from the task, not the framework. The paper's ten case studies show that real applications mix patterns, and the authors state that no single pattern is likely to suffice across all threat models.

Use case	Main risk	Patterns that fit	Supporting controls
Customer service chatbot	Off-topic use, injected instructions in the user message	Action-selector, context-minimisation	Fixed tool set, no free-form outbound actions
Email and calendar assistant	Incoming email instructs the agent to forward data	Plan-then-execute or code-then-execute, dual LLM	User confirmation before sending, recipient allowlist
Product recommender or review summary	One poisoned review steers the whole answer	LLM map-reduce	Numeric or boolean outputs per item
CV screening	Hidden text in a CV ranks it first	Map-reduce, dual LLM for summaries	Human review of shortlist
SQL or data analysis agent	Data in tables alters queries; code execution escapes	Plan-then-execute	Read-only database role, sandboxed interpreter
Software engineering agent	Instructions in repos, issues and dependencies	Dual LLM with strict data formats	Action sandboxing, scoped repository tokens
Medical or regulated information bot	Answers drift beyond approved content	Context-minimisation	Answers sourced only from approved documents

Small teams or first agent: pick action-selector or plan-then-execute with a short tool allowlist. They are easy to build and to explain to reviewers.
Agents over many untrusted documents: map-reduce first, because it bounds each document's influence and parallelises well.
Open-ended assistants with tools: dual LLM or a CaMeL-style capability layer, plus human approval for anything in the Rule of Two danger zone.
Coding and computer-use agents: these break most patterns because the task needs all three Rule of Two properties. Rely on sandboxing, scoped tokens and approval gates. See the AI Agent Sandboxing and Code Execution Guide.

Anthropic's engineering guidance on building agents makes a related point: use the simplest workflow that works (prompt chaining, routing, parallelisation, orchestrator-workers, evaluator-optimiser) before reaching for a fully autonomous loop. A fixed workflow is already a partial plan-then-execute and is far easier to secure.

Where identity fits

A design pattern decides which component may request a tool call. Identity decides what that call can reach. The two must be designed together, or the pattern's guarantee leaks out through an over-privileged credential.

Separate identities per component: the privileged planner, quarantined readers and map workers should not share one API key. Give the quarantined LLM no tool credentials at all. See the Agent Identity and Access Guide.
Short-lived, scoped tokens: issue per-task, per-tool tokens (OAuth 2.0 token exchange or workload identity) rather than long-lived secrets in the agent's environment. See the Credential Vaulting for AI Agents Guide.
Act on behalf of the user, not above them: the agent's effective permissions should be the intersection of its own role and the requesting user's rights, so injected instructions cannot reach data the user could not.
Policy outside the model: authorisation decisions belong in a policy engine at the tool gateway, which is the point CoSAI's workstream 4 makes in its work on agentic identity and access management. See the AI Agent Authorisation Guide.

How to implement

Inventory and classify: for each agent, list every input source and tag it trusted (the authenticated user, system configuration) or untrusted (web, email, documents, tool results, other agents, memory). List every tool and tag it read, write or external communication.
Apply the Rule of Two: mark which of [A], [B] and [C] each session needs. Where all three are present, redesign or add an approval step before writing code.
Pick and document the pattern: record in the design document which pattern applies, where the trust boundary sits and which component sees untrusted text. Reviewers should be able to trace any tool call back to trusted input.
Enforce at the tool boundary: put a gateway or orchestrator between the model and tools that checks each call against the plan or allowlist, validates argument schemas, blocks unapproved destinations (URLs, email domains, repositories) and attaches the scoped credential.
Close exfiltration channels: strip or proxy markdown images and links in output, enforce an egress allowlist and keep content security policy allowlists current. The EchoLeak 2025 and ForcedLeak 2025 cases both leaked data through image URLs on trusted domains.
Test in shadow mode first: run the constrained agent alongside the existing workflow, log plan deviations and blocked calls, and run an injection test suite (indirect injection in every untrusted source) before granting write tools. See the AI Agent Security Testing Checklist.
Roll out by ring: internal users, then a pilot group, then general availability, enabling write and external tools last. Keep a feature flag that drops the agent back to read-only or action-selector mode.
Keep a rollback and kill switch: revoke the agent's tokens centrally and disable tools at the gateway without a code release. See the AI Agent Kill Switch and Containment Guide.

Detection and measurement

Patterns are only as good as the evidence that they hold. Log at the orchestrator, not just the model API.

Log: the trusted request, the committed plan or generated program, every untrusted item retrieved with its source, every tool call with arguments, the credential used, policy decisions (allow, block, escalate) and approval outcomes.
Alert on: tool calls not in the committed plan, calls to destinations outside the allowlist, quarantined output that fails schema validation, credentials used outside their task, and spikes in blocked calls from one source.

Metric	What it shows	Target direction
Share of agents with a documented pattern and trust boundary	Design coverage	Towards 100% for agents with write tools
Injection test pass rate per release	Whether the pattern holds under attack	Rising, no regression between releases
Plan-deviation and blocked tool call rate	Attempts the deterministic layer caught	Investigated, not ignored
Sessions combining all three Rule of Two properties without approval	Residual high-risk autonomy	Zero
Median lifetime of agent tool credentials	Blast radius of a leaked token	Minutes to hours, not months
Task success rate after constraints	Utility cost of the pattern	Tracked so security changes are deliberate

Framework mapping

Framework	Reference	How the patterns support it
OWASP Top 10 for Agentic Applications (2026)	ASI01 Agent Goal Hijack, ASI02 Tool Misuse and Exploitation, ASI03 Identity and Privilege Abuse, ASI05 Unexpected Code Execution	Control-flow isolation counters goal hijack; tool gateways and scoped tokens counter misuse and privilege abuse; sandboxing for code-then-execute
OWASP Top 10 for LLM Applications 2025	LLM01 Prompt Injection, LLM06 Excessive Agency	Patterns reduce the impact of injection and the agency available to injected text
MITRE ATLAS	AML.T0051, AML.T0053, AML.T0086, AML.T0057 LLM Data Leakage	Threat model inputs for design review and testing
NIST SP 800-53 Rev. 5	AC-4 Information Flow Enforcement, AC-6 Least Privilege, SC-7 Boundary Protection, SI-10 Information Input Validation, AU-12 Audit Record Generation, CA-8 Penetration Testing	Trust boundaries, scoped tools, egress control, argument validation, orchestrator logging, injection testing
NIST CSF 2.0	PR.AA-05, DE.CM-09	Least-privilege permissions for agents; monitoring of runtime environments
ISO/IEC 27001:2022 Annex A	8.27 Secure system architecture and engineering principles, 8.28 Secure coding	Design patterns as documented engineering principles
CIS Controls v8.1	Control 16 Application Software Security	Secure design and testing of in-house agent applications

Common failure patterns

Relying on the system prompt: "ignore instructions in documents" is not a control. EchoLeak bypassed a production injection classifier and still reached data through an image reference.
Trusted channels that are not trusted: allowlisted domains, link previews and content security policy entries become exfiltration paths. In ForcedLeak an expired allowlisted domain was re-registered for about five dollars.
Leaking untrusted text back to the planner: a dual LLM design fails the moment a convenience feature passes the quarantined model's free text, rather than a variable or a constrained value, to the privileged model.
Plans that are not enforced: the agent writes a plan, but the orchestrator executes whatever the model emits next. The plan must be checked by code at each step.
One credential for everything: a broadly scoped API key shared by all agent components turns any successful injection into full access. Gemini CLI's 2025 flaw showed how an agent's own shell access becomes the attacker's: see the Gemini CLI Prompt Injection Flaw 2025.
Ignoring memory and other agents as untrusted inputs: retrieved memories and messages from other agents carry injected text just like web pages. See the AI Agent Memory Poisoning Guide.

Practitioner checklist

Classify every input source of each agent as trusted or untrusted, including tool results, memory and other agents.
Run the Agents Rule of Two in design review and require approval or redesign where all three properties meet.
Document the chosen pattern and trust boundary for every agent with write or external tools.
Enforce plans, tool allowlists and argument schemas in a gateway outside the model.
Give each agent component its own short-lived, scoped credential, and none to quarantined readers.
Block markdown image and link exfiltration and review egress and CSP allowlists for expired domains.
Run indirect injection tests against every untrusted source before each release and track pass rates.
Log plans, retrieved items, tool calls and policy decisions, and alert on plan deviations.
Keep a tested kill switch that revokes agent tokens and disables tools without a deployment.

Secure Agent Design Patterns Guide: Plan-Then-Execute, Dual LLM and Other Injection-Resistant Architectures