AI instruction hijacking exposes a runtime control gap in LLMs

At a glance

What this is: LEVO’s analysis says AI instruction hijacking is a runtime authority failure in which lower-trust instructions can override system constraints inside LLMs.

Why it matters: This matters because enterprises connecting LLMs to data and tools need governance over instruction hierarchy, context assembly, and action boundaries, not just safer prompts.

Context

AI instruction hijacking is a control problem, not a simple content problem. In layered LLM systems, system directives, developer rules, retrieved context, and user input are all interpreted together, which means authority has to be preserved by design rather than assumed. That matters for identity and access governance because the model may be given access to data or tools without an enforceable runtime boundary around who or what can influence its decisions.

The failure mode appears when untrusted content is able to reshape model behavior without ever becoming a legitimate authority source. In practice that means prompt injection, retrieval poisoning, and session memory drift can all create the same outcome: the model continues to operate, but under manipulated instruction precedence. For AI security, the governance question is no longer whether the model can answer, but whether its runtime instructions remain trustworthy.

Key questions

Q: What breaks when instruction hierarchy is not enforced in an LLM?

A: The model can treat lower-trust text as if it had equal or greater authority than system constraints. That breaks policy boundaries, weakens safety rules, and can redirect tool use or data access. The core failure is not that the model reads untrusted content, but that the application fails to preserve authority relationships at runtime.

Q: Why do prompt injection attacks create governance risk for AI agents?

A: Prompt injection creates governance risk because the model often sits in the control path between text input and tool execution. If attackers can change what the model treats as authoritative, they can influence access decisions, data exposure, or downstream actions without compromising a traditional account. That makes prompt provenance and instruction hierarchy part of AI identity governance.

Q: How do security teams know whether LLM firewall controls are working?

A: They should look for blocked secrets, logged policy violations, session-linked traces, and a measurable reduction in unsafe prompts and leaked outputs. If the team cannot reconstruct who submitted what, what the model returned, and which policy intervened, the control is not operationally complete.

Q: How should organisations govern tool use when LLMs can act on instructions?

A: Treat every tool call as an authorization event, not a language event. The model may suggest or justify an action, but the application should verify that the action fits policy, scope, and context before execution. This is especially important when prompts can be shaped by retrieved documents or persistent memory.

Technical breakdown

Why instruction hierarchy in LLMs is implicit, not enforced

Large language models do not execute instructions the way deterministic software executes code. System messages, developer rules, retrieved text, and user prompts are concatenated into one context window, then interpreted probabilistically. The model may learn patterns that resemble priority, but there is no native execution engine that hard-codes precedence between trusted and untrusted sources. That makes instruction integrity a design property rather than a built-in control. In governance terms, authority is only real if the surrounding application enforces it before the model acts.

Practical implication: treat instruction precedence as a runtime control requirement, not as an assumed model behaviour.

How prompt injection becomes AI instruction hijacking

Prompt injection is the technique of introducing adversarial instructions into the model context. Hijacking happens only when those instructions succeed in changing behaviour, such as weakening policy constraints, redefining task scope, or influencing tool use. The distinction matters because many attacks attempt injection, but only some achieve authority reversal. Once that reversal occurs, the model may still look functional while operating under redirected intent. That is why this is best understood as a control-plane compromise rather than a simple bad-output event.

Practical implication: evaluate success conditions, not just attack attempts, when assessing LLM control failure.

Why retrieval layers and session memory expand the attack surface

Retrieval-augmented generation and persistent memory introduce additional instruction sources that can be mistaken for trusted context. A poisoned document can carry directive language that influences reasoning, and earlier turns can continue shaping later responses even after system prompts are repeated. Because the model blends these sources semantically, the attack surface extends beyond the user prompt itself. In agentic workflows, that becomes more serious because manipulated instructions can influence tool invocation, workflow execution, or data access, not just text generation.

Practical implication: inspect retrieval and memory paths as part of your AI access-control boundary.

Threat narrative

Attacker objective: The attacker’s objective is to redirect the model’s authority so it produces unauthorized outputs or actions while appearing to operate normally.

1. Entry occurs when adversarial or untrusted instructions are introduced through user input, retrieved documents, uploaded content, or persisted session memory.
2. Credential_harvested is not the main pattern here; the equivalent failure is authority capture, where the lower-trust instruction source gains functional precedence over system intent.
3. Escalation follows when the model reinterprets the blended context and adopts the manipulated directive as the effective behavioural authority.
4. Impact appears when the hijacked instruction drives policy violations, restricted data exposure, or unauthorized tool invocation in connected enterprise workflows.

Breaches seen in the wild

• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.
• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI instruction hijacking is a runtime control-plane problem, not a prompt-quality problem. If a model can be influenced by lower-trust text at the moment it decides how to behave, then the organisation does not have instruction governance, only content moderation. That distinction matters because enterprise risk emerges when the model is connected to sensitive data or operational tools. The practitioner conclusion is straightforward: secure the decision path, not just the input surface.

Instruction hierarchy is the missing named control concept here: runtime instruction integrity. The article’s central insight is that system authority can be preserved in design language but lost in execution if the application does not enforce precedence before inference. That is especially relevant for AI governance teams because model behaviour becomes a governance issue only when authority is measurable at runtime. The practitioner conclusion is to make instruction precedence auditable, not aspirational.

Prompt injection, RAG poisoning, and session memory drift are different entry points into the same failure mode. The article shows that the threat is not limited to one exploit string or one vulnerable workflow. Once retrieved context or persistent memory can compete with system instructions, the model’s behavioural boundary becomes unstable. The practitioner conclusion is to treat every context source as part of the AI trust boundary.

Tool use makes instruction hijacking materially more dangerous than text manipulation alone. When a hijacked instruction can trigger an API call, update a record, or move a workflow forward, the issue crosses from model integrity into operational authority. That is why AI security and identity governance now overlap. The practitioner conclusion is to bind tool execution to policy-aware controls, not just model output checks.

AI security programmes need a specific control concept for this risk: instruction provenance. Enterprises need to know which instruction sources were present, how they were weighted, and what the model actually treated as authoritative. Without that visibility, post-incident review will miss the reason the model changed behaviour. The practitioner conclusion is to instrument context assembly the same way identity teams instrument privilege and session activity.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: Agentic AI Security Guide

What this signals

Runtime instruction integrity is now a control requirement for AI programmes. The article shows that models can remain operational while operating under redirected authority, which means the real governance problem is not output quality alone but whether the instruction stack is trustworthy at the moment of decision. Teams that already manage identity, access, and privilege should recognise the pattern: authority must be explicit, observable, and revocable.

AI instruction hijacking widens the boundary between AI security and identity governance. Once a model can invoke tools, access retrieval layers, or move work through a session, it behaves like a governed actor even if it is not a human identity. That means AI programmes need controls for instruction provenance, tool authorization, and context separation before they scale agentic workflows.

AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. For practitioners, that reinforces the same message as the article: the model is only one part of the trust boundary, and the surrounding orchestration layer often carries the sharper exposure.

For practitioners

• Enforce runtime instruction precedence Separate system, developer, retrieved, and user instructions in the application layer so conflicting content cannot silently compete inside one undifferentiated prompt.
• Inspect retrieval paths for directive content Scan RAG sources, uploaded documents, and knowledge bases for language that attempts to steer behaviour, redefine policy, or override task boundaries.
• Constrain tool invocation to policy checks Require explicit authorization rules before the model can call APIs, update records, or trigger workflows, even when the prompt appears to justify the action.
• Instrument instruction provenance and drift Log which context sources were present, how memory changed across turns, and where output behaviour diverged from the intended authority order.

Key takeaways

• AI instruction hijacking is what happens when untrusted instructions successfully override higher-authority system intent at runtime, leaving the model operational but behaviourally redirected.
• The article’s key warning is that prompt injection, RAG poisoning, and memory drift are different entry paths into the same governance failure, so controls must cover the whole instruction stack.
• Practitioners need runtime visibility into context assembly and tool authorization, because instruction hierarchy is only real when the application enforces it before the model acts.

Key terms

• AI Instruction Hijacking: A runtime failure where lower-trust instructions override higher-authority system intent inside an LLM. The model continues to operate, but its behaviour is redirected by manipulated context, which can lead to policy violations, disclosure, or unsafe tool use.
• Instruction Integrity: Instruction integrity is the property that operational guidance remains untampered and trustworthy as it moves through retrieval, orchestration, and model execution. In AI systems, losing instruction integrity can cause constraint override, unsafe tool use, or data exposure even when the underlying infrastructure is authenticated.
• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Retrieval-augmented Generation: Retrieval-augmented generation is a pattern where an AI model pulls external information before generating output. The security challenge is that access rules can weaken when data is chunked, embedded, cached, or reused, so source permissions may not automatically follow the content into the model's context.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and secrets management. It helps security practitioners translate identity control thinking into AI runtime governance.