Prompt injection security: why testing and runtime controls matter

At a glance

What this is: Panther argues that prompt injection is a security vulnerability in LLM systems, not just an AI safety issue, because it can drive data theft, privilege abuse, and harmful actions through trusted workflows.

Why it matters: It matters because IAM, PAM, and NHI teams now have to govern model input, tool access, and delegated action paths as part of the control plane, not treat them as prompt hygiene.

Context

Prompt injection is a control problem that appears when an LLM cannot reliably separate trusted instructions from untrusted content. In practice, the attack surface widens as soon as the system can retrieve documents, call tools, or act on behalf of a user, which makes this a governance issue for AI-enabled access paths as well as a model-safety issue.

The article’s central point is that prompt-only defenses do not hold once the model can move from text generation to execution. That is where the identity angle becomes material: if an agent can inherit permissions, call APIs, or send data onward, then authorization scope and runtime monitoring become part of the security design, not afterthoughts.

Key questions

Q: What breaks when prompt injection can trigger agent actions directly?

A: What breaks is the separation between input handling and privileged execution. Once malicious content can drive tool use, file changes, or command execution, the organisation no longer has a safe boundary between reading untrusted data and acting on it. Controls must stop the transition, not just detect the source text.

Q: When should teams prioritise runtime monitoring over prompt-only defenses?

A: Teams should prioritise runtime monitoring as soon as a model can retrieve content, use tools, or act on behalf of a user. Prompt-only defenses cannot keep pace with changing inputs, obfuscation, and new retrieval sources. Monitoring becomes essential when the model’s output can trigger side effects that matter to security and business operations.

Q: What are the signs that an agent has drifted outside its intended scope?

A: Common signs include repeated attempts to reach restricted resources, unusual sequencing of tools, requests that expand beyond the original task, and behavior that bypasses normal review points. A more subtle signal is when the agent keeps making progress by exploring paths the operator did not authorize. Teams should watch the entire trajectory, not just isolated tool calls.

Q: How should security teams handle prompt injection in AI systems?

A: Treat prompt injection as an authorisation problem, not only a content problem. Validate user input, monitor for override patterns, and block any prompt that tries to change model policy, exfiltrate data, or trigger sensitive actions. The safest design is one where untrusted prompts cannot directly reach code execution, production data, or privileged tools without policy checks.

Technical breakdown

Direct and indirect prompt injection attack paths

Prompt injection usually arrives through two mechanisms. Direct injection targets the model’s live input and tries to override system instructions. Indirect injection hides malicious instructions inside content the model later retrieves, such as support tickets, documents, web pages, or code comments. The risk is higher in retrieval-augmented and agentic systems because the model may treat attacker-controlled content as if it were part of the task context. Once the model is allowed to act, the same instruction can move from misleading text to unsafe tool use.

Practical implication: Separate trusted instructions from retrieved content and test every input channel that can influence model behaviour.

Why agentic AI turns prompt injection into delegated abuse

Agentic systems raise the stakes because the model is no longer only generating responses. It can call tools, move data, send messages, or trigger downstream actions, so a successful injection becomes a path into delegated privilege abuse. In identity terms, the model behaves like a non-human actor with an action surface that must be scoped and monitored. The security question is not whether the model can be persuaded, but whether a persuaded model can reach a sensitive sink such as email, code execution, or external transmission.

Practical implication: Treat agent permissions as a least-privilege problem and remove high-impact tools from the default action path.

Why guardrails work better as detection than as prevention

Guardrails can reduce risk, but they are not reliable enough to serve as the only barrier. Language models are stochastic, attack patterns change quickly, and obfuscation can defeat simple filters. The stronger approach is to use guardrails to classify, score, and contain suspicious inputs or outputs while the surrounding system enforces hard controls on tools, destinations, and side effects. That means prevention sits in architecture and authorization, while the guardrail layer becomes a tripwire and evidence source.

Practical implication: Use guardrails to detect and contain suspicious behaviour, but enforce hard limits outside the model.

Threat narrative

Attacker objective: The attacker wants to convert model trust into unauthorized data movement or action execution through the AI system’s own delegated permissions.

1. Entry occurs when an attacker submits a malicious prompt directly or hides one inside retrieved content that the model will process later.
2. Credential or tool access follows when the model inherits the ability to call APIs, send data, or trigger actions beyond the original user intent.
3. Escalation happens when the compromised prompt path causes the model to disclose sensitive context, invoke unsafe tools, or execute code in agentic workflows.
4. Impact is data exfiltration, unauthorized action, or remote code execution depending on what the model was allowed to do.

Breaches seen in the wild

• Meta Muse agent hijack 2026: An undocumented Muse setting let local malware hijack Meta's personal AI agent, steal its authentication material and abuse user access.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Prompt injection is now an identity and access problem, not only a model-safety problem. Once an LLM can call tools or act on behalf of a user, the real issue becomes who or what is allowed to authorise those actions. That shifts the control surface toward permissions, delegation, and runtime scope. Teams that treat the model as a passive text system will miss the governance failure mode. The practitioner conclusion is to govern AI action paths with the same discipline used for privileged access.

Runtime control matters because pre-deployment testing cannot freeze the attack surface. New prompts, retrieval sources, tools, and model versions all change the security boundary after release. That means continuous testing and monitoring are part of the control plane, not a one-off validation step. The practical lesson is to build a living assurance loop around agent behaviour, especially where tool use or outbound communication is possible.

Blast-radius control is the decisive security variable for agentic systems. The article shows that the dangerous combination is private data access, untrusted content exposure, and outbound communication capability. If any one of those is removed, the attack becomes less viable. This is the named concept to carry forward: prompt-injection blast radius is the amount of damage a successful injection can do once the model has delegated access. Practitioners should reduce that radius before they rely on any detection layer.

AI security teams need to test the sink, not just the prompt. A payload that lands but cannot change state is far less dangerous than one that reaches email, files, or code execution. That means the security objective is to understand which actions are reachable from each input path. The conclusion for governance is straightforward: map sources, sinks, and allowed tools before you certify an AI workflow for production.

Agent identity and permission ownership must be explicit. The article’s monitoring guidance only works if teams know which agent is allowed to do what, and under which context. Without that, logs become noise and policy checks lose meaning. For identity and AI governance teams, the implication is to assign clear ownership for agent permissions, response boundaries, and tool access before scaling AI assistants.

What this signals

Prompt-injection blast radius: this is the amount of damage a successful injection can cause once a model has permission to act. For practitioners, the point is to reduce reachable side effects before relying on monitoring or guardrails, because the model’s permissions define the real risk envelope.

AI workflows should be treated as live control paths that change every time prompts, tools, or retrieval sources change. That means assurance has to be continuous, with re-testing and monitoring tied to release management rather than a one-time AI safety review.

For practitioners

• Map trust boundaries before testing Document every input source as trusted or untrusted, then list the tools, APIs, email paths, and code execution surfaces each model can reach. Use that map to define the model’s real blast radius rather than assuming the system prompt creates a boundary.
• Build a versioned adversarial test suite Maintain a library of direct, indirect, obfuscated, multi-turn, RAG-poisoning, and multimodal prompt injection patterns. Run the suite in CI/CD on every build and fail changes that let the model comply with malicious instructions.
• Reduce agent privileges to the minimum Remove unnecessary write actions, separate high-risk tools from low-risk tools, and require human approval for actions with side effects such as sending email, writing files, or invoking external APIs.
• Instrument runtime monitoring for scope deviations Log tool calls, destination domains, guardrail triggers, and trace IDs so you can detect when model behaviour departs from the approved action profile. Alert on external exfiltration paths, unexpected tool use, and repeated guardrail hits across sessions.
• Treat guardrails as containment layers Use input and output classifiers to flag suspicious content before and after the model processes it, but do not rely on them as the only control. Pair them with hard policy enforcement outside the model runtime.

Key takeaways

• Prompt injection is an operational security issue because it can convert untrusted text into unauthorized model behaviour, including data exposure and unsafe actions.
• The evidence in the article shows that indirect attacks, agent tooling, and retrieval paths create a larger blast radius than simple chat prompts alone.
• The strongest control pattern is layered governance: shrink permissions, test adversarially in CI/CD, and monitor runtime tool use for scope deviations.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions, including calling APIs, writing code, and orchestrating other agents, with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
• Blast Radius: The potential scope of damage if a specific credential or identity is compromised. Identities with broad permissions have a larger blast radius and represent a higher priority for least-privilege enforcement and security controls.
• Guardrails: Guardrails are policy controls that inspect prompts and model outputs against defined safety, privacy, and compliance rules. In AI operations, they reduce harmful language and disclosure risk, but they do not replace entitlement management, logging, or identity governance for the systems that call the model.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management. It helps practitioners connect identity controls to the broader security programmes that AI and automation now depend on.