Prompt injection security: why testing and runtime controls matter

At a glance

What this is: Panther argues that prompt injection is a security vulnerability in LLM and agentic systems, not just an AI safety issue, and lays out testing, hardening, and monitoring practices.

Why it matters: For IAM, PAM, and NHI teams, the issue is that AI agents and LLM workflows inherit identity and privilege problems as soon as they can retrieve data or take actions.

By the numbers:

• 72% of organisations have experienced or suspect they have experienced a breach of non-human identities, 46% confirmed and 26% suspected.

👉 Read Panther's blog on testing prompt injection and reducing risk

Context

Prompt injection is a weakness in LLM applications where untrusted content can be interpreted as instructions, especially when the system retrieves data or can trigger downstream actions. In practice, the security problem is not the text itself but the trust boundary between developer instructions, user input, retrieved context, and tools. For identity teams, that boundary becomes relevant the moment an AI system acts with credentials, tokens, or delegated access.

Panther’s article frames the risk correctly: once an LLM can read internal documents, call APIs, send email, or execute code, injection becomes a privilege problem as much as a model problem. That makes the control stack familiar to identity practitioners. Least privilege, isolation, and runtime monitoring are the same ideas, but applied to AI workflows instead of human users or service accounts.

Key questions

Q: How should security teams handle prompt injection in production LLM applications?

A: Security teams should treat prompt injection as a runtime control issue, not a content-moderation problem. The practical response is to inspect prompts, retrieved content, and tool outputs for hostile instructions, then block or downgrade unsafe sessions before the model can act on them. The model should never be the only enforcement layer.

Q: Why do AI agents make prompt injection more dangerous than chat-only tools?

A: AI agents are more dangerous because they can act, not just generate text. When a model can invoke tools, access records, or send messages, a hidden instruction can become a real enterprise action. The risk rises sharply if the agent inherits broad NHI permissions instead of narrowly scoped access.

Q: What do teams get wrong about guardrails for prompt injection?

A: They treat guardrails as a prevention layer instead of a detection and containment layer. Filters help, but attackers can bypass them with obfuscation, context manipulation, or payload variation. A better approach is to assume some injections will succeed and then limit what the model can do, detect scope deviations, and isolate high-risk actions.

Q: Who is accountable when an AI agent takes an unsafe action?

A: Accountability should sit with the business owner of the agent, the team that provisioned the access, and the control owners responsible for monitoring and revocation. If no one can answer who approved the identity, the scope, and the oversight model, the governance framework is not complete enough for production.

Technical breakdown

Direct and indirect prompt injection paths

Direct prompt injection happens when an attacker places malicious instructions into the model’s immediate input. Indirect prompt injection is more dangerous because the payload hides in retrieved or external content that the model later treats as context, such as documents, tickets, code comments, or web pages. The model cannot reliably distinguish instructions from data by itself, which is why retrieval pipelines create an attack surface. When agents can act on what they read, the exploit chain moves from misinformation to operational abuse.

Practical implication: Map every input source by trust level before deploying AI features and treat retrieved content as untrusted by default.

Why agentic AI turns text attacks into privilege issues

Agentic systems raise the stakes because the model is no longer only generating text. It can now call tools, access data, send messages, or trigger workflows, which means a successful injection can produce side effects outside the chat window. The real risk is the combination of untrusted input plus a capable sink, such as email, code execution, or external API access. In identity terms, the system has acquired a runtime privilege set that attackers can try to steer.

Practical implication: Restrict tools, isolate high-risk actions, and require human approval for anything that changes state or crosses a trust boundary.

Guardrails and monitoring as detection layers

Prompt filters and response classifiers help, but they do not solve prompt injection because language models are probabilistic and attack patterns evolve quickly. Panther’s article is strongest when it treats guardrails as containment and detection, not prevention. That is the right mental model for AI security and for NHI governance around AI systems. Production telemetry should include tool calls, destination domains, scope deviations, and repeated guardrail triggers so teams can see when the system steps outside its intended role.

Practical implication: Instrument LLM workloads with structured logging and alert on tool calls, destination changes, and repeated guardrail events.

Threat narrative

Attacker objective: The attacker wants to turn an AI feature into a controllable action path that leaks data, abuses tools, or executes unintended commands.

1. Entry occurs when malicious instructions are submitted directly through the chat interface or indirectly through retrieved content such as documents, code comments, or support tickets.
2. Escalation occurs when the model accepts attacker-controlled instructions as context and uses its available tools, credentials, or integrations to follow them.
3. Impact occurs when the agent exfiltrates information, performs unauthorized actions, or executes code in a way that crosses the system’s intended privilege boundary.

NHI Mgmt Group analysis

Prompt injection is an identity and privilege problem once AI systems can act. The article is right to treat prompt injection as a security vulnerability, but the deeper governance issue is that agentic workflows inherit access control assumptions from IAM and NHI programmes. Once an AI system can read internal data, call tools, or send output externally, the question becomes who or what is authorised to do those things and under which constraints. Practitioners should govern the agent as a runtime identity, not just as a model.

Blast radius, not prompt purity, is the decisive control variable. The article correctly notes that no single hardening method is foolproof. That means the programme objective shifts from preventing every injection to limiting what the system can do after one succeeds. Read-only access, isolated execution, and high-friction approval for side effects are the controls that change outcomes. Practitioners should measure the damage window, not just the detection score.

Indirect prompt injection creates a retrieval trust gap that many teams still underestimate. External content is often treated as business data, but in an LLM pipeline it can function as attacker-controlled input. That is the same structural mistake that has long affected email, web, and document workflows, now reproduced inside AI. For identity and data teams, this is where RAG governance, access scoping, and content provenance must converge.

Named concept: retrieval-to-tool privilege chain. This is the failure mode where untrusted retrieved content influences a model that can then trigger tools or external actions. The chain matters because the exploit is not limited to text manipulation; it can become downstream state change. Practitioners should design controls around the full path from retrieval to action, not only the prompt surface.

Detection must move from content inspection to scope deviation monitoring. The most useful signals in the article are tool calls, domain allowlists, and repeated guardrail triggers because they show when the system behaves outside its declared role. That maps well to broader security governance: focus on runtime behaviour, not just content classification. Practitioners should make scope deviation a first-class control objective.

What this signals

Prompt injection is forcing security teams to think beyond content moderation and into runtime authorisation for AI workflows. The practical signal for programmes is that AI systems need the same kind of scoped access, monitoring, and approval boundaries that identity teams already apply to privileged service accounts and automated workloads.

Retrieval-to-tool privilege chain: the next control gap will be whether organisations can govern the path from untrusted retrieval to external action. That means aligning AI monitoring with established identity governance patterns, while also using the NIST Cybersecurity Framework 2.0 to anchor detection, response, and recovery expectations.

Teams should expect their first AI incidents to look less like model failure and more like authorization failure. That shifts the programme conversation toward tool allowlists, runtime telemetry, and explicit ownership for agent permissions, which is where NHI governance and AI governance start to overlap.

For practitioners

• Map trust boundaries for every AI input path Classify system prompts, user prompts, retrieved context, and tool outputs separately, then document which sources are trusted and which are not. Include the APIs, mail systems, and external services the model can reach so blast radius is explicit.
• Minimise the agent’s effective privilege set Use read-only connections where possible, separate high-risk tools from low-risk tools, and require human approval before any action that sends data, writes records, or executes code. Treat the model as a constrained runtime identity.
• Build adversarial tests into CI/CD Maintain a versioned corpus of direct injection, indirect injection, obfuscation, multi-turn escalation, RAG poisoning, and multimodal cases, then fail the build if the model complies with any of them.
• Monitor runtime scope deviations Log tool calls, destination domains, guardrail triggers, and request traces so production detections can alert on behaviour outside the approved agent scope. Use the same approach for AI workflows that you would use for privileged service accounts.

Key takeaways

• Prompt injection is a security vulnerability with identity and privilege consequences once an AI system can retrieve data or take actions.
• Testing, hardening, and monitoring must work together because prompt-only defenses cannot reliably stop adaptive attacks.
• The key governance question is not whether an attack is possible, but how much access the model should have when one succeeds.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads — causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Indirect Prompt Injection: Indirect prompt injection is an attack where malicious instructions are hidden inside content that an AI system reads later. The model may treat that content as context rather than as hostile input, which can influence tool use, data access, or workflow actions if controls are weak.
• Agent Scope Manifest: An agent scope manifest is a versioned description of what tools, destinations, and actions an AI agent is allowed to use. It gives security teams a concrete baseline for detecting scope deviation and for limiting the blast radius when model behaviour drifts or is manipulated.
• Scope Deviation: Scope deviation is any action an AI system takes outside its approved role, such as calling an unapproved tool, reaching an external domain, or writing data when only read access was intended. It is one of the clearest runtime signals that an AI workflow has been steered or misused.

What's in the full article

Panther's full blog covers the operational detail this post intentionally leaves for the source:

• Versioned attack-library structure for direct, indirect, and multimodal prompt injection tests
• Practical detection ideas for guardrail triggers, tool-call anomalies, and system prompt leakage
• Implementation notes for integrating probe tools such as garak and promptfoo into CI/CD
• Examples of detection-as-code workflows for AI-specific security monitoring

👉 Panther's full post covers the test patterns, hardening steps, and monitoring signals in more detail

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management for practitioners who need to govern runtime access. It helps security teams connect identity controls to modern automation, agentic workflows, and privileged access decisions.