Prompt injection is exposing the limits of legacy AI security

At a glance

What this is: Prompt injection is a growing AI security problem where malicious instructions can hide inside normal content and steer LLMs or agents into unsafe behaviour.

Why it matters: It matters because IAM, NHI and AI governance teams increasingly need to control what agents can read, invoke and act on, not just who can log in.

Context

Prompt injection is a governance problem, not just a content-filtering problem. Large language models do not reliably separate instructions from data, so malicious text can be embedded in prompts, documents, emails, web pages or tool outputs and then acted on by an AI system. That breaks the assumptions behind legacy security controls that were built for code, malware or static policy enforcement rather than probabilistic model behaviour.

The identity angle becomes material as soon as an AI system can call tools, access data or act on behalf of a user or service. In that environment, the control question shifts from whether an input is allowed to whether the model, agent or connected NHI can be steered into using legitimate permissions in unsafe ways. That makes prompt governance, tool authorisation and runtime oversight part of the identity security problem space.

Key questions

Q: What breaks when prompt injection reaches a tool-using AI agent?

A: What breaks is the assumption that the model's output is low impact. Once the agent can call tools, a malicious instruction can become a database query, a file write, an email, or a deployment action. Without policy checks and approval gates, the agent's legitimate permissions become the attacker's path to impact.

Q: Why do agentic apps make prompt injection more dangerous than chatbots?

A: Agentic apps can turn manipulated text into real action. A chatbot can produce a bad answer, but an agent can query systems, send messages, create files, or execute code under its own credentials. That means the blast radius is determined by authorization and runtime policy, not by the text output alone.

Q: What are the signs that AI prompt controls are not working?

A: Common signs include employees copying sensitive data into consumer AI tools, false positives from regex-based DLP, and repeated workarounds after controls block legitimate prompts. Another indicator is when the organisation can inventory AI tools but cannot say which sensitive data has actually been exposed. That means governance stops at visibility, not enforcement.

Q: How should security teams govern AI prompts that include sensitive data?

A: Treat the browser as a control point, not just an interface. Inspect the sensitivity of the data, the identity of the user, and the context of the session before the prompt leaves enterprise control. That lets teams allow useful AI use while blocking risky disclosure paths without relying only on after-the-fact DLP.

Technical breakdown

Why prompt injection bypasses classic input security

Prompt injection works because LLMs do not enforce a hard boundary between command and content. In a conventional application, a query parser or sanitizer can separate code from data. In an LLM, the same text stream may contain instructions, context, metadata and user content, so a malicious phrase inside a document or message can be interpreted as an operational directive. That means the attack surface is not a single field or endpoint, but every place the model reads text or processes multimodal content. The security problem is semantic, not syntactic.

Practical implication: control what the model is allowed to consume, not just what the user is allowed to submit.

Why agentic tool use expands the blast radius

Once an LLM can invoke tools, APIs or MCP-connected services, a successful prompt injection can move from bad output to bad action. The model may use legitimate credentials, approved integrations or delegated access to perform an unsafe call, retrieve data or change a workflow. That is why agentic systems create a stronger governance requirement than standalone chat interfaces. The risk is not that the model merely says the wrong thing. It is that it can be steered into exercising existing authority in ways the operator never intended.

Practical implication: treat every tool invocation as privileged behaviour that requires policy and audit coverage.

Why legacy security stacks miss AI-native abuse

Firewalls, DLP, IDS/IPS, endpoint tools and WAFs are built to detect known payloads, malicious binaries or suspicious network and application events. Prompt injection is neither a normal malware pattern nor a predictable exploit string. It often appears as ordinary language, and its effect depends on model behaviour at runtime. That is why AI-native controls focus on adversarial testing, prompt hardening, monitoring model outputs and tracing tool use. In practice, the security boundary has moved into the model interaction layer, where traditional scanners have little visibility.

Practical implication: add AI-specific runtime monitoring and adversarial testing to your control stack rather than extending only perimeter controls.

Threat narrative

Attacker objective: The attacker wants to steer an AI system into revealing data or executing unsafe actions through trusted permissions rather than exploiting code directly.

1. Entry begins when malicious instructions are embedded in a prompt, document, email, web page or other content the model will process.
2. Credential or authority abuse occurs when the model or agent treats the hidden instructions as part of its task and uses legitimate access to call tools or reach data.
3. Impact follows when the agent leaks information, overrides system instructions, or performs unauthorised actions that were enabled by its delegated permissions.

Breaches seen in the wild

• EchoLeak (Microsoft 365 Copilot) 2025: A crafted email could make Microsoft 365 Copilot leak data from its context with no click, a zero-click prompt injection fixed as CVE-2025-32711.
• ForcedLeak (Salesforce Agentforce) 2025: A critical Agentforce flaw let attackers steer an AI agent through a Web-to-Lead form to leak CRM data via an expired allowlisted domain.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Prompt injection is the AI-era equivalent of semantic trust abuse. The security failure is not simply malicious text, but the model's willingness to treat untrusted language as operational intent. That makes the core governance problem closer to identity abuse than to traditional malware detection. Practitioners should assume that any model with tool access can be manipulated unless input, context and action are governed together.

Tool governance becomes the real control plane for agentic AI. Once an agent can act through APIs, MCP servers or delegated workflows, the meaningful security question is whether those actions are least-privileged, observable and revocable. The article's central warning is that legacy controls rarely understand model intent or tool invocation paths. Teams need to manage AI permissions as carefully as service account permissions.

Shadow AI turns prompt injection into an inventory problem as well as a runtime problem. Organisations cannot secure what they cannot see, and undiscovered models, agents and plugins create blind spots that no perimeter tool can close. This is where NHI governance intersects with AI governance: every agent, connector and machine credential should be owned, scoped and monitored. The practitioner conclusion is that AI discovery now belongs in identity governance workflows.

Prompt injection will force a shift from blocking inputs to governing outcomes. Defenders have spent years hardening code paths against injection, but AI systems respond to context rather than syntax. That means the security objective is no longer simply preventing a bad prompt from arriving. It is proving that an AI system cannot turn a bad prompt into a privileged action, a data leak or a policy breach.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

Prompt injection will increasingly be governed like privilege abuse, not just content abuse. Security leaders should expect the control conversation to move toward who can let an agent read, remember and act, because that is where the attack becomes operational. The practical response is to put AI permissions, logging and exception handling into the same governance cycle used for other high-risk identities.

Semantic attack surfaces require a different operating model from malware detection. A malicious prompt does not need to look like code to create harm, so detection has to follow model behaviour, tool usage and unexpected downstream actions. That is why runtime visibility and adversarial testing belong in the same programme, especially where agents interact with privileged data or workflows.

Prompt injection should be treated as a trigger to formalise AI governance debt. If an organisation has deployed copilots, agents or MCP-connected tools without an inventory, approval model or monitoring discipline, the exposure compounds quickly. The emerging standard is not simply to block bad prompts, but to prove that AI systems cannot convert untrusted language into unauthorised action.

For practitioners

• Inventory every model, agent and connector Map where LLMs, chatbots, copilots, plugins and MCP servers exist across business units, including shadow deployments and embedded features.
• Restrict tool and MCP permissions to least privilege Limit each agent or model to the smallest set of APIs, workflows and data sources needed for its task, and separate read from write access wherever possible.
• Add adversarial prompt testing to release gates Test for direct injection, indirect injection, jailbreak patterns, role overrides and obfuscated instructions before models or agents are promoted to production.
• Monitor agent output and action paths at runtime Log model responses, tool calls, delegated actions and unusual escalation patterns so suspicious behaviour can be detected and blocked during execution.

Key takeaways

• Prompt injection exploits a model's inability to cleanly separate instructions from data, which makes legacy perimeter controls unreliable for AI systems.
• The risk escalates sharply when models can call tools or agents can act through delegated permissions, because malicious prompts can become real actions.
• Effective defence requires AI-native governance across inventory, least-privilege tool access, adversarial testing and runtime monitoring.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions, including calling APIs, writing code, and orchestrating other agents, with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Runtime Guardrail: A control applied while an AI agent is operating, not just during configuration or review. Guardrails can block dangerous tool calls, require approval for sensitive actions, or stop data leakage before it reaches systems or users.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security and secrets management. It helps practitioners connect identity controls to the broader governance demands created by AI agents and automated workflows.