Prompt injection works by shifting the model’s frame, not breaking it

At a glance

What this is: This analysis explains why prompt injection succeeds in LLMs: attackers can shift the model’s frame by changing context, not by exploiting a parser.

Why it matters: It matters because LLM security controls that focus only on malicious text detection can miss frame-shift attacks, leaving agentic and human-facing AI workflows exposed to instruction override.

Context

Prompt injection is an LLM governance problem, not a classic code-injection problem. The model receives instructions, user text, and retrieved context through the same channel, so attackers can alter the effective frame of the conversation without breaking a parser or crossing a hard boundary.

That matters for GenAI-enabled applications, autonomous assistants, and support workflows because the control failure sits at the level of context handling and instruction priority. Once untrusted content can compete with trusted instructions inside the context window, prompt filters alone stop being a reliable security boundary.

Key questions

Q: What breaks when prompt injection is not governed like an access problem?

A: The organisation may treat malicious text as a harmless message, even though it can steer an agent into exposing data or taking privileged actions. Prompt injection is dangerous because it turns untrusted content into a control plane for behaviour. Teams need policy and authorisation checks around outputs, not just message filtering.

Q: Why do frame-shift attacks create more risk in agentic AI workflows?

A: Agentic systems can read mail, process documents, and call tools, so a successful frame shift can become an action rather than just a bad answer. Once the model accepts the attacker’s frame, the downstream tool chain can amplify the damage across data access, automation, and external side effects.

Q: What are the signs that prompt filtering is failing?

A: Common signals include hidden Unicode characters, suspicious formatting, unexpected role shifts, cross-language instruction fragments, and payloads that only become harmful when split across turns. If the system behaves safely on isolated inputs but not on assembled context, the weakness is usually in prompt normalisation or context handling.

Q: Should teams prioritise output constraints or intent detection for LLM safety?

A: Output constraints should come first when the application handles untrusted context or can trigger tools. Intent detection is useful, but it remains brittle because the same words can be safe in one frame and unsafe in another. Structural limits reduce harm even when the model is successfully steered.

Technical breakdown

Why LLMs treat instructions and data as one context stream

Traditional injection exploits a parser boundary that separates code from data. LLMs do not have that boundary in the same way. They consume tokens as context, and the model infers which instruction set is active from pattern and framing rather than from a guaranteed source of truth. That means user text, retrieved documents, and system instructions can compete inside one context window. The security problem is not merely harmful content. It is that the model lacks an external trust oracle for provenance. When the frame changes, the continuation distribution changes with it, which is why the same request can be refused in one setting and answered in another.

Practical implication: Treat prompt text as untrusted input and move enforcement outside the model where possible.

How authority transfer and genre shifting steer model behaviour

Prompt injection often works by changing who the model thinks it is talking to and what kind of interaction is underway. Authority transfer uses roles such as auditor or researcher to move the model into a peer-to-peer expert frame. Genre shifting places harmful content inside a fiction, training, or tabletop scenario frame, where describing an attack looks like legitimate task performance. The model does not verify whether the social context is real. It responds to the language game suggested by the prompt. That is why compounded framing is effective: several weak shifts can combine into a strong enough signal to override the safer default frame.

Practical implication: Review prompts for stacked framing cues, not just explicit malicious keywords.

Why classifiers miss indirect prompt injection

Classifier-based defences usually inspect either the input or the output surface after the model has already accepted the frame. That is too late when the harmful instruction is buried in a document, email, ticket, or other retrieved artefact. Indirect prompt injection works because the model cannot reliably distinguish a trusted wrapper from an attacker-controlled payload once both are present in context. A safety layer that only scores direct harmful requests will underperform when the request is embedded in legitimate workflow language. The deeper issue is architectural: the model can be redirected into compliance before the detector has a stable signal to act on.

Practical implication: Validate retrieved content and constrain model actions before untrusted text reaches decision-making steps.

Threat narrative

Attacker objective: The attacker wants the model to ignore its safer instruction set and produce harmful, misleading, or disclosure-prone output as if the request were legitimate.

1. Entry occurs when attacker-controlled text enters the context window through a prompt, ticket, document, or retrieval source.
2. Credential access is not the primary mechanism here; the attacker instead abuses instruction priority and framing to make the model treat the malicious text as authoritative.
3. Impact occurs when the model follows the shifted frame and produces unsafe output, leaks sensitive content, or performs unintended actions through downstream tools.

Breaches seen in the wild

• ForcedLeak (Salesforce Agentforce) 2025: A critical Agentforce flaw let attackers steer an AI agent through a Web-to-Lead form to leak CRM data via an expired allowlisted domain.
• Meta Muse agent hijack 2026: An undocumented Muse setting let local malware hijack Meta's personal AI agent, steal its authentication material and abuse user access.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Prompt injection is fundamentally a governance failure about frame control, not content moderation. The model is not being tricked by a bad string alone. It is being induced to adopt a different conversational frame in which unsafe behaviour becomes coherent. For NHI and agentic AI programmes, that means the control question is who or what gets to set the active frame, not merely whether the text looks suspicious. Practitioner conclusion: security teams need enforcement points outside the model.

Compounded framing is the specific failure mode defenders are missing. A single authority cue may be weak, but authority transfer, genre shift, and embedded context can reinforce each other until the model treats attacker instructions as the most plausible continuation. That makes prompt injection more like social engineering for models than a text-pattern problem. Practitioner conclusion: assess how multiple benign-looking cues combine inside one workflow, not just whether a prompt contains hostile keywords.

Frame-shift attacks expose the limits of prompt-layer safety in agentic systems. If an AI agent can read mail, summarise tickets, or trigger tools, the harmful outcome may occur even when the model never outputs obviously toxic text. This is where AI agent identity and authorisation intersect with prompt security. Practitioner conclusion: pair prompt hardening with scoped tool permissions and output constraints.

Architectural control, not classifier confidence, is the durable security boundary. Output schemas, restricted tool access, and explicit instruction hierarchy do more than detect bad behaviour. They reduce the set of actions the model can take after a frame shift succeeds. Practitioner conclusion: treat prompt filters as advisory and enforce the real boundary in the application layer.

Frame control is becoming a distinct AI governance concept. Organisations will need to think about which prompts, retrieved documents, and upstream roles are allowed to reshape model behaviour inside operational workflows. That is a governance issue for AI systems and, increasingly, a lifecycle issue for non-human identities that act on untrusted context. Practitioner conclusion: inventory the places where context can override policy.

From our research library:

• Generative AI use specifically increased from 33% in 2023 to 79% in 2025, according to McKinsey’s Global Surveys on the State of AI.
• Read next: Agentic AI Security Guide

What this signals

Frame control is becoming a first-order AI security control. Teams that rely on prompt filtering alone are already behind the threat model, because the attack works by reshaping the model’s active context rather than by forcing a parser failure. The practical shift is toward controlling what untrusted content can enter the model’s decision path at all.

Prompt injection and agent abuse sit on the same governance fault line. Once an LLM can read retrieved documents or act on behalf of a user, attacker-controlled context can become non-human identity abuse through delegated action. The boundary you need to secure is the one between untrusted input and authorised execution, not the prompt text alone.

Generative AI use specifically increased from 33% in 2023 to 79% in 2025, according to McKinsey’s Global Surveys on the State of AI, which means far more organisations now need frame-aware controls before these workflows become normalised.

For practitioners

• Define frame-control checkpoints Map where authority transfer, genre shifting, and embedded context can alter model behaviour in your highest-risk workflows. Put review points around retrieval, ticketing, and user-supplied documents before those inputs reach the model.
• Constrain model output by design Use structured output formats, allowlisted actions, and explicit instruction priority so the model cannot freely emit unsafe prose or tool calls after a prompt shift.
• Scope agent permissions tightly Limit what AI agents can read, write, and execute when they operate on untrusted context, especially in email, support, and document-processing workflows.
• Test against compounded prompt attacks Red-team with stacked authority, genre, and embedded-context prompts, not just direct harmful requests, because the model may fail only when multiple cues reinforce one another.

Key takeaways

• Prompt injection succeeds because LLMs infer authority from context, so the same text can steer the model differently depending on framing and workflow.
• The core risk is not a broken parser but a weakened trust boundary, which makes prompt-only defences unreliable in systems that ingest untrusted documents or tickets.
• Architectural controls such as scoped tool access, structured outputs, and context validation are the practical limiters when model behaviour can be reshaped at runtime.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Frame Shifting: The practice of changing the model’s perceived role, genre, or audience so a different response pattern becomes more likely. In security terms, it is a way to make unsafe output look coherent within a new context, which can reduce the effectiveness of naive refusal logic.
• Credential Transfer: Credential transfer is the controlled movement of passwords, passkeys, or other secrets from one provider to another. In practice, it should reduce manual file handling, preserve structure, and limit the time secrets spend outside governed systems.
• Extension Context: The runtime scope in which a VS Code extension operates. It determines what APIs, storage, and events the extension can access while it is running. In security terms, context is meant to separate one extension’s state from another, but that separation only holds if the platform enforces it consistently.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is suited to practitioners who need to connect access governance to emerging machine identity and AI agent risk.