Why do prompt injection attacks remain hard to stop?

Prompt injection is hard to stop because the model may not reliably distinguish trusted instructions from malicious content once both are present in the same context. Defenses must therefore be layered, including input filtering, context isolation, tool restrictions, and adversarial testing. A prompt policy alone does not provide runtime assurance.

Why This Matters for Security Teams

Prompt injection remains hard to stop because it is not a classic malware problem. The malicious instruction can arrive in a webpage, email, document, ticket, chat thread, or retrieved knowledge source, then compete with the system prompt and user request inside the same context window. That makes it a governance and runtime isolation problem, not just a content-filtering problem. Guidance from OWASP Agentic AI Top 10 and MITRE ATLAS both reflect that adversarial inputs can redirect model behaviour even when the model appears to be following policy.

The security impact is broader when the model can call tools, access secrets, or act on behalf of a user. A successful injection may cause data exfiltration, fraudulent actions, or unsafe automation without any obvious compromise of the underlying infrastructure. NHIMG research on OWASP NHI Top 10 shows that agentic systems increase the blast radius because identity, privilege, and context are now part of the attack surface. In practice, many security teams encounter prompt injection only after a model has already followed an attacker’s hidden instruction, rather than through intentional testing.

How It Works in Practice

Prompt injection succeeds when an attacker can place untrusted text where the model treats it as operationally relevant. That can happen through direct user prompts, but the more difficult cases involve indirect injection: content pulled from a browser, RAG corpus, support ticket, or shared workspace that is later processed as if it were trusted. The model does not reliably enforce a hard boundary between data and instructions, so defenders must create that boundary outside the model.

Current practice usually combines several controls:

• Separate trusted instructions from retrieved or user-supplied content as strictly as possible.
• Restrict tool use by default, especially for file access, outbound requests, and secret retrieval.
• Validate outputs before action, rather than assuming the model’s response is safe.
• Log prompts, tool calls, and retrieval sources for investigation and red-teaming.
• Test with adversarial examples mapped to MITRE ATLAS adversarial AI threat matrix and the MITRE ATT&CK Enterprise Matrix where agent actions overlap with cyber kill chains.

For teams operating agents, the identity layer matters as much as the prompt layer. If an AI agent can inherit a privileged token, the injection no longer needs to “hack” the model in a traditional sense; it only needs to steer a legitimate identity into unsafe action. NHIMG’s LLMjacking research highlights how quickly exposed credentials are abused in the wild, reinforcing why agent credentials must be tightly scoped and short-lived. These controls tend to break down when the model is allowed to browse, summarize, and act on the same untrusted context without a separation layer because the attacker’s text becomes operational input.

Common Variations and Edge Cases

Tighter prompt and tool controls often increase engineering overhead and reduce model flexibility, so organisations need to balance safety against usability and automation value. There is no universal standard for this yet, and best practice is still evolving, especially for multi-agent workflows and long-context systems.

One common edge case is RAG. If retrieval quality is poor, malicious or stale content can outrank the intended policy and still influence the model. Another is multilingual or obfuscated injection, where the attacker hides instructions inside formatting, encoded text, or content that looks like metadata. A third is agent chaining, where one model’s output becomes another model’s instruction, amplifying risk across systems. NHIMG’s Key Challenges and Risks material is useful here because it shows how hidden trust relationships create failure modes that are not obvious from the application layer alone.

For governance and assurance, the right baseline is to treat prompts as untrusted input unless they are generated by a controlled system component, and to apply runtime monitoring in line with the NIST SP 800-53 Rev 5 Security and Privacy Controls. The hardest cases are systems that mix open-ended retrieval, autonomous tool execution, and weak privilege boundaries, because a single injection can move from influence to action without a second approval step.

Related resources from NHI Mgmt Group

• What is the difference between prompt injection and traditional injection attacks?
• Should organisations rely on model safety features alone to stop prompt injection?
• Why are AI gateways not enough to stop prompt injection and data leakage?
• Why do prompt injection attacks create governance risk for AI agents?