Prompt injection exposes the trust model behind LLM applications

At a glance

What this is: This is a practical guide to prompt injection in LLM applications, showing that the core weakness is the lack of a reliable boundary between trusted instructions and untrusted text.

Why it matters: It matters because IAM, PAM, and application teams have to govern LLM tools as privilege-bearing systems, not just treat prompt filtering as a content problem.

By the numbers:

• Prompt injection has been ranked the number one vulnerability on the OWASP Top 10 for LLM Applications since 2025.
• Research cited in the article says as few as five strategically poisoned documents in a RAG knowledge base can manipulate AI responses 90% of the time.
• The article says joint research from OpenAI, Anthropic, and Google DeepMind found sophisticated attackers can bypass published defenses with over 90% success rates when given enough attempts.

Context

Prompt injection is a security problem in LLM applications where untrusted text is treated as if it were instruction. The issue is not just malicious wording, but that current model architectures blend system prompts, user input, and retrieved content into one context without a reliable trust boundary.

For identity teams, the risk is governance failure as much as content manipulation. Once an LLM can read data, call tools, or act on behalf of a user, prompt injection becomes an access-control problem, because the model may operate with permissions wider than the input that triggered it.

That makes this a control-plane issue for NHI, agentic AI, and application security programmes. The article's examples show that the most dangerous failures happen when language models are coupled to tools, credentials, and workflow automation with weak privilege boundaries.

Key questions

Q: What breaks when prompt injection controls are not tested against instruction hierarchy failures?

A: When instruction hierarchy is not tested, a model may follow user content over system or developer instructions, especially if malicious instructions are embedded in inputs or retrieved data. That can lead to unsafe tool calls, data exposure, policy bypass, and uncontrolled agent actions. Testing must confirm that higher priority instructions consistently override untrusted input in real operating conditions.

Q: Why do prompt injections become more dangerous when LLMs can use tools?

A: Tool access turns a bad output into an operational action. If the model inherits the user's permissions, a successful injection can write files, send messages, modify settings, or exfiltrate data, so the real risk is privilege misuse rather than text manipulation alone.

Q: How do teams know whether prompt injection controls are actually working?

A: Look for end-to-end visibility across prompts, retrieved content, memory, tool calls, and outputs, plus evidence that blocked actions stay blocked under realistic test cases. If the system can only be evaluated with static prompts, the controls are probably too narrow. Behaviour drift under multi-turn workflows is the signal to watch.

Q: What should security teams do first when indirect prompt injection is a concern?

A: Start by mapping the full agent trace, not just the prompt. Identify every place untrusted text can enter the reasoning path, then restrict the tools and data sources reachable from that path. If you only scan user input, you will miss the channels that matter most in production agent workflows.

Technical breakdown

Why prompt injection defeats instruction boundaries

Prompt injection works because the model does not enforce a hard separation between executable instructions and data. System prompts, retrieved documents, and user messages are all presented as text in one context window, so the model can be persuaded to follow attacker-supplied instructions over developer intent. That is different from parameterised SQL, where the engine keeps code and data separate. In LLM systems, the model itself is the parser, so trust is inferred from context rather than enforced by a privilege boundary. This makes the failure architectural, not just behavioural.

Practical implication: design around hard trust boundaries in the application layer, not around assumptions that the model will reliably distinguish instruction from content.

How indirect and multimodal injection expands the attack surface

Indirect injection hides malicious instructions in content the model ingests from emails, documents, web pages, or RAG sources. Multimodal injection extends that pattern into images, audio, and video, where the payload may be embedded in metadata, pixel data, or transcription artifacts. The model then processes the hidden instruction as contextual input. The security problem grows when external content is mixed with privileged workflows, because the user never has to explicitly type the attack. That is why content provenance and isolation matter as much as prompt hygiene.

Practical implication: tag, isolate, and constrain untrusted content before it enters the model context, especially in retrieval and multimodal pipelines.

Why agentic tool use turns prompt injection into privilege abuse

When an LLM can call tools, write files, change settings, or send messages, prompt injection stops being only a text-in, text-out problem. The model may inherit the user's permissions and then take actions that the user never intended. That is an identity issue because the effective actor is no longer just the human, but the human-plus-agent execution path. This is where least privilege, scoped tokens, approval gates, and deterministic allowlists become part of the security architecture rather than optional safeguards.

Practical implication: treat every tool-bearing LLM as a delegated identity and constrain its permissions to the smallest executable scope.

Threat narrative

Attacker objective: The attacker wants to make the model ignore its intended instructions and use delegated access to disclose data, modify state, or execute actions on the victim's behalf.

1. Entry occurs through direct user input, hidden instructions in retrieved content, or payloads embedded in documents, emails, images, or code comments that the model ingests.
2. Credential or action access is gained when the model accepts the injected instruction as part of its context and routes it into tool use, workspace changes, or data access.
3. Escalation happens when the model inherits the user's permissions and applies them to actions the user did not authorise, including file writes, settings changes, and exfiltration.
4. Impact is unauthorised disclosure, code execution, or workflow abuse through the LLM application's tool chain and delegated access path.

Breaches seen in the wild

• ForcedLeak (Salesforce Agentforce) 2025: A critical Agentforce flaw let attackers steer an AI agent through a Web-to-Lead form to leak CRM data via an expired allowlisted domain.
• EchoLeak (Microsoft 365 Copilot) 2025: A crafted email could make Microsoft 365 Copilot leak data from its context with no click, a zero-click prompt injection fixed as CVE-2025-32711.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 150+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Prompt injection is a delegated access problem, not a prompt-quality problem. The article's central point is that LLMs process mixed-trust text as one context, which means instruction boundaries are not enforced in the way security teams expect. Once tools and data access are attached, the attack becomes a governance failure over delegated execution, not just a content-safety issue. Practitioners should treat LLMs as privilege-bearing systems and not as neutral text processors.

Language models turn identity scope into attack scope. When an agent acts with the permissions of the user it serves, any prompt injection inherits that user’s access boundaries. That means the blast radius is determined by the delegated identity, not by the text channel alone. The practical implication is that access governance, approval gates, and scope design now sit inside the LLM control surface.

Language is the weak trust boundary in current LLM architectures. The model cannot reliably tell whether a sentence is an instruction or data, so the trust model assumes a separation that does not exist. That assumption fails under direct, indirect, and multimodal injection because the attacker can smuggle control instructions through ordinary content. Practitioners should recognise this as a structural control gap, not a tuning issue.

Fine-grained authorization is now a primary AI security control. The article correctly elevates least privilege, scoped tokens, and human confirmation because tool access is where prompt injection becomes operational harm. This is where OWASP-AGENTIC, OWASP-NHI, and NIST AI governance expectations converge: the model's output matters less than the action path it can trigger. Teams should measure whether agent permissions are narrower than the user's real business need.

Ephemeral credential trust debt: LLM systems accumulate risk whenever short-lived context, retrieved data, and delegated permissions outlive the moment they were safe to use. That debt shows up when a model can still act on stale or poisoned context after the user has moved on. Security teams should design around immediate privilege scoping and revocation at execution time, not after the fact.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: Agentic AI Security Guide

What this signals

Prompt injection creates a governance problem that traditional content filters cannot solve. As LLMs move from chat interfaces into tool-bearing workflows, the relevant question is not whether the text looks malicious but whether the surrounding identity and authorization model can limit what the model is allowed to do.

Language-model trust assumptions are now a security design decision. Teams that bind prompts to short-lived, scoped credentials and explicit approval gates reduce the operational blast radius when injected context reaches the model. That is especially important when untrusted content and business actions share the same workflow.

Ephemeral credential trust debt: The longer a model can carry context, the more opportunity an attacker has to poison it, and the harder it becomes to reason about what should still be trusted. The article's example set shows why prompt hygiene alone does not scale to agentic execution.

For practitioners

• Treat external content as untrusted input Classify user messages, retrieved documents, emails, and tool outputs as data, never as instructions. Keep control text server-side and separate trusted system guidance from content that can be poisoned by an attacker.
• Constrain delegated model privileges Issue narrowly scoped, short-lived credentials to any LLM or agent that can call tools or act on behalf of a user. Limit action scope to the smallest business need and require explicit approval for destructive or high-impact operations.
• Add deterministic tool guardrails Enforce hard allowlists and parameter validation in application code so the model cannot choose unsafe calls on its own. Block sensitive actions even when the model output appears confident or well-formed.
• Run adversarial testing against LLM workflows Red-team direct, indirect, and multimodal prompt injection paths on a recurring schedule. Test both the model and the surrounding workflow, including retrieval, file writes, and action execution.
• Instrument model inputs and outputs Log prompts, tool calls, and high-risk responses so security teams can detect unusual instruction patterns, unexpected tool use, and data exposure events during incident review.

Key takeaways

• Prompt injection is an architectural trust failure in LLM systems, not just a content moderation problem, because the model cannot reliably separate instructions from untrusted text.
• The article shows why tool-bearing LLMs are especially risky, with injected context able to turn delegated permissions into unauthorised actions, data exposure, or code execution.
• The most effective controls are privilege minimisation, explicit approval for sensitive actions, deterministic guardrails, and adversarial testing across the full workflow.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Indirect Prompt Injection: Indirect prompt injection is an attack where malicious instructions are hidden inside content that an AI system reads later. The model may treat that content as context rather than as hostile input, which can influence tool use, data access, or workflow actions if controls are weak.
• Agentic Tool Execution: Agentic tool execution is the pattern where an AI system does more than generate text and instead calls APIs, reads data, runs functions, or changes state. This creates security risk because legitimate operations can be manipulated into unintended actions, making intent harder to distinguish from normal behavior.
• Delegated Identity: Delegated identity is when one actor acts on behalf of another with explicit permission and bounded authority. In AI-assisted commerce, it requires clear consent, limited scope, and traceable records so the retailer can distinguish authorised delegation from unauthorised automation.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.