Malicious prompt injections are expanding the AI attack surface

At a glance

What this is: This is a Polymer analysis of malicious prompt injections and the way they expand the AI threat surface by manipulating generative systems into unsafe behaviour.

Why it matters: It matters because identity, access, and data controls must now account for AI systems that can be influenced at runtime, especially where prompts, tools, and delegated actions intersect.

👉 Read Polymer's analysis of malicious prompt injections in enterprise AI workflows

Context

Malicious prompt injection is a control problem as much as a model problem. Once generative AI is embedded into workflows across collaboration, SaaS, and productivity tools, attackers can use crafted instructions to bypass guardrails and steer outputs or actions. The security gap appears when organisations treat AI output as advisory while the system is also able to invoke tools or surface sensitive data.

For IAM, NHI, and AI governance teams, the relevant question is who or what is authorised to act on a prompt, a tool call, or a downstream workflow step. If delegated AI behaviour is not tied to explicit identity, policy, and logging controls, the organisation inherits a shadow decision layer that traditional access review processes were not designed to govern.

Key questions

Q: What breaks when an AI system cannot separate instructions from data?

A: The trust boundary breaks first, then the policy boundary follows. A retrieved document, email, or webpage can be interpreted as an instruction instead of evidence, which lets adversaries influence model behavior without ever touching the user interface. Once that happens, traditional keyword filters and prompt rules become incomplete because they are defending the wrong layer.

Q: Why do AI assistants create a new trust problem for identity governance?

A: AI assistants create a new trust problem because they can read data, choose tools, and act on external text in ways traditional review processes do not expect. Identity governance has to account for action promotion, provenance, and egress, not only authentication or entitlement assignment.

Q: How do security teams know runtime AI guardrails are actually working?

A: Look for blocked poisoned inputs, flagged anomalous outputs, and traceable enforcement before responses reach users or downstream systems. If controls only inspect prompts or only inspect outputs, they leave a gap that attackers can exploit through manipulated data sources or tool responses.

Q: Should organisations treat AI plugins like privileged access?

A: Yes. AI plugins and connected APIs can act on behalf of the model, so they should be governed as privileged access paths with narrow scopes, explicit approval boundaries, and continuous review. If a plugin can reach customer records or production systems, it belongs in the same governance conversation as PAM and NHI controls.

Technical breakdown

How malicious prompt injection manipulates model behaviour

A malicious prompt injection embeds conflicting or hidden instructions inside content the model ingests, such as a document, message, webpage, or chat thread. The model then follows the higher-priority instruction hierarchy or semantic cues in ways the user did not intend. In practice, the risk is amplified when the LLM is wrapped in an agentic workflow that can retrieve data, call tools, or draft responses without strong policy checks. The weakness is not that the model “gets hacked” in a classic sense, but that the surrounding system trusts unverified instructions too much.

Practical implication: separate user intent, retrieved content, and executable tool instructions into distinct policy layers.

Why AI tool use turns prompt injection into an access problem

Once an AI system can act through APIs, SaaS connectors, or internal tools, prompt injection becomes an authorisation issue. The model may not hold credentials directly, but the surrounding agent often does, which means hostile instructions can ride on a trusted execution path. This is especially relevant for NHI governance because service tokens, scoped API keys, and delegated permissions may be exposed to actions triggered by untrusted text. In other words, the security boundary shifts from model output to what the system is allowed to do after interpreting that output.

Practical implication: treat every tool-capable AI workflow as a privileged execution path with explicit allow lists and scoped secrets.

How guardrails fail when the content source is trusted too early

Many AI safety designs focus on prompt filters, content moderation, or post-generation checks, but those controls can fail if untrusted content is processed before policy enforcement. If the system retrieves from documents, emails, or external sources and then lets the model reason over that content without sanitisation, the malicious instruction has already entered the decision chain. The same pattern appears in enterprise data security: trust in source reputation is not a substitute for instruction boundary control. The problem is less about one poisoned prompt and more about an unsafe control plane for AI inputs.

Practical implication: validate and classify inputs before retrieval, reasoning, and tool execution, not only after output is generated.

Threat narrative

Attacker objective: The attacker aims to hijack AI-assisted workflows so the system reveals data, takes unintended actions, or undermines trust in automated decisions.

1. Entry occurs when an attacker places malicious instructions into content that an AI system will later ingest, such as shared documents, messages, web pages, or knowledge base entries.
2. Escalation occurs when the model follows those hidden instructions inside a workflow that has access to tools, data connectors, or delegated actions.
3. Impact occurs when the AI leaks data, changes decisions, or performs unsafe actions through trusted integrations under the organisation's own permissions.

NHI Mgmt Group analysis

Malicious prompt injection is an instruction-boundary failure, not just a model-safety issue. The core problem is that enterprises often let untrusted content enter the same reasoning path as trusted prompts and business instructions. That collapses the separation between data, policy, and execution, which is why retrieval, summarisation, and tool use can all become attack surfaces. Practitioners should treat instruction boundaries as a governance control, not a prompt-engineering detail.

AI workflows are becoming non-human actors with delegated power. Once an LLM can choose tools, query systems, or trigger actions, it behaves like a governed runtime entity rather than a passive application feature. That brings NHI-style questions into AI security: what identity does the workflow use, who owns it, what can it access, and how is its behaviour revoked when conditions change. Teams should map agent permissions to the same lifecycle discipline used for other privileged non-human identities.

Shadow AI expands when tool access is easier to add than to govern. Many organisations can bolt an LLM onto existing applications faster than they can define approval, logging, and containment rules for its actions. That creates hidden operational risk because the model may appear harmless while the connected workflow has broad data and system reach. Practitioners should expect the biggest failures where AI adoption outruns identity governance and access review processes.

Prompt injection exposes a new kind of governance debt. Existing security programmes often assume that the dangerous part is the model output, but in agentic environments the dangerous part is the full chain from instruction ingestion to tool execution. The named concept here is instruction-boundary collapse, meaning the organisation no longer has a reliable line between content that should influence language and content that should influence action. Teams should build controls that preserve that boundary before scaling AI automations.

The identity of AI systems now matters as much as the identity of users. If a workflow can read, decide, and act, it needs ownership, scope, monitoring, and revocation just like any other privileged runtime identity. That does not mean every LLM is autonomous, but it does mean every action-capable deployment becomes an identity governance problem. Practitioners should align AI control design with IAM and NHI lifecycle management rather than treating it as a standalone safety layer.

What this signals

AI security programmes are moving into the same governance territory as IAM and NHI because action-capable models now sit inside access paths, not just content generation paths. That means the control question is no longer whether the model is accurate enough, but whether the workflow is authorised, observable, and revocable across its full execution path.

Instruction-boundary collapse: when untrusted content can influence both reasoning and action, the organisation loses the line between information and authorisation. Teams should plan for policy enforcement before retrieval and before tool execution, then connect that design to [OWASP NHI Top 10](https://nhimg.org/complete-guide-to-the-2026-owasp-top-10-risks-for-agentic-applications) patterns where agent misuse becomes a practical risk.

The programmes most exposed are those that add AI connectors faster than they define runtime ownership. That is the same structural weakness seen in unmanaged third-party access, where delegated trust outpaces lifecycle governance and the result is a hidden privilege surface.

For practitioners

• Isolate prompt, content, and action paths Keep untrusted content out of the same control path that authorises tool calls. Separate retrieval, model reasoning, and execution so a malicious instruction cannot become an approved action by default.
• Scope every AI tool credential tightly Bind API keys, service accounts, and tokens to the minimum data sources and operations each AI workflow needs. Rotate and revoke those credentials with the same discipline used for other privileged non-human identities.
• Log AI inputs and tool decisions together Record the originating content, the prompt context, the model response, and any downstream action in one audit trail. Without this correlation, incident responders cannot tell whether the model, the content source, or the integration was the failure point.
• Review AI workflows as privileged runtimes Classify any AI system that can trigger actions, query sensitive data, or write to production systems as a high-risk runtime identity. Require ownership, approval, and periodic revalidation before expanding its scope.

Key takeaways

• Malicious prompt injection becomes dangerous when AI systems can turn untrusted content into action, not just text.
• The governance gap is the loss of a clear boundary between data, instruction, and execution across AI workflows.
• Security teams should govern action-capable AI like privileged non-human identities, with scoped access, auditability, and revocation.

Key terms

• Malicious Prompt Injection: A malicious prompt injection is content designed to influence an AI system’s instructions, outputs, or actions in ways the user did not intend. It works by exploiting the model’s trust in retrieved or embedded text, especially when the system can call tools or access sensitive data.
• Instruction Boundary Collapse: Instruction boundary collapse occurs when a system cannot reliably distinguish trusted operator direction from malicious content embedded in data it reads. For AI agents, this matters because emails, tickets, webpages, and metadata can all become carriers for commands if the agent processes them as instructions.
• Action-capable AI Workflow: An action-capable AI workflow is a system where a model can do more than generate text and can instead retrieve data, invoke tools, or trigger business processes. That makes it a privileged runtime that needs ownership, scope, logging, and revocation controls.
• Delegated Machine Action: Delegated machine action is work performed by an AI agent under authority inherited from a human or system sponsor. The identity remains non-human, but the accountability path still traces back to the original delegate. In practice, this makes runtime behaviour, not just issuance, the governance concern.

What's in the full article

Polymer's full blog post covers the operational detail this post intentionally leaves for the source:

• Practical examples of how malicious prompt injections enter enterprise workflows through common SaaS and collaboration paths
• Operational discussion of how data security controls and content filtering can be layered before model reasoning and tool execution
• Implementation context for teams that need to decide where prompt controls end and access governance begins
• The article's broader data-security framing for organisations using generative AI in production

👉 Polymer's full post covers the AI threat model, workflow exposure points, and the governance implications for security teams

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, IAM, and machine identity security for practitioners who need to govern delegated access safely. It helps security leaders connect identity lifecycle controls to the broader systems now carrying privileged machine action.