TL;DR: Prompt injection lets adversarial input override a generative AI model’s intended instructions, exposing data, triggering unauthorized actions, and undermining trust in GenAI systems, according to Lasso Security. The control problem is not just malicious text, but the assumption that untrusted input can be safely mixed with instructions.
At a glance
What this is: This is a guide to prompt injection in GenAI, showing how malicious inputs can override instructions and drive disclosure or unauthorized actions.
Why it matters: It matters because teams building GenAI into business workflows need to govern prompt trust boundaries, output handling, and control separation before those systems can safely touch sensitive data or actions.
Context
Prompt injection is a GenAI security problem where untrusted input is interpreted as instruction, not just data. That breaks the normal assumption that a model can reliably separate user content, system prompts, and downstream actions.
For IAM and security teams, the issue is not just model output quality. Once a GenAI system can reveal sensitive data or execute actions on behalf of a user, prompt handling becomes an access-control and trust-boundary problem, not only an application-security one.
Key questions
Q: What breaks when an AI chatbot can treat untrusted text as an instruction?
A: The main failure is that the chatbot stops being a responder and starts acting like a control surface. When output can be reused as input, attackers can redirect formatting, external links, or tool calls. That turns a conversation into workflow manipulation and can expose sessions, accounts, or internal actions.
Q: Why do prompt injection attacks create security risk beyond bad model outputs?
A: Because many GenAI deployments connect model responses to tools, workflows, and data systems. If the response is allowed to trigger real actions, a successful injection can alter records, reveal internal details, or cause other operational harm. The risk rises when organisations treat the model as a trusted decision point instead of an untrusted intermediary.
Q: What are the signs that a GenAI application is being misused or attacked through prompt injection?
A: Common warning signs include the model producing unexpected outputs, retrieving data outside the current user’s scope, or invoking tools that were not intended for the request. Other signals are unusual prompt size, repeated attempts to steer the model, and actions that expose sensitive data or credentials. These behaviors suggest the application boundary is being manipulated.
Q: How should organisations govern GenAI systems that interact with users in real time?
A: They should treat GenAI as a live governed service, not a static model. That means defining approved use cases, assigning accountable owners, testing for misuse before release, and monitoring behaviour after deployment. The strongest programmes connect policy, red teaming, and incident response so that safety controls evolve as prompts, data sources, and workflows change.
Technical breakdown
How prompt injection overrides system prompts
Prompt injection works because large language models process text in a single conversational context unless developers deliberately separate instruction layers. An attacker can place malicious instructions inside apparently harmless content, and the model may follow the injected directive over the intended system prompt. That is why direct and indirect injection both matter: one arrives through the chat channel, the other through content the model later reads. The technical weakness is not just “bad text”; it is a prompt architecture that does not reliably preserve instruction precedence.
Practical implication: isolate control instructions from untrusted content and assume prompt text cannot be trusted by default.
Why untrusted input can trigger unauthorized actions
GenAI systems become riskier when they are connected to tools, workflows, or backend actions. In that setup, a manipulated prompt can do more than produce a wrong answer. It can request data access, alter records, or drive other actions the surrounding application is prepared to execute. The model itself is not the only issue. The real exposure appears when application logic accepts the model’s output as if it were an authorised command, especially in customer service, document processing, and administrative agents.
Practical implication: treat model output as untrusted until a separate policy layer validates the action and its scope.
Why direct and indirect prompt injection require different defenses
Direct injection happens when an attacker speaks to the model directly, while indirect injection hides malicious instructions inside emails, documents, web pages, or other data sources the model consumes. That distinction matters because the attack surface changes with the data flow. A chatbot needs different controls than an assistant that reads inboxes or internal files. If every source can become instruction-bearing content, then the environment needs filtering, logging, sandboxing, and strong separation between retrieved data and executable prompts.
Practical implication: map every model input path and apply source-specific filtering and containment controls to each one.
Threat narrative
Attacker objective: The attacker wants to make the GenAI system obey hostile instructions so it leaks data, performs unauthorized actions, or produces harmful content.
- Entry occurs when an attacker supplies malicious prompt text directly to a chatbot or hides it inside content the model will later consume.
- Credential or data exposure follows when the model treats the injected content as instruction and reveals hidden prompts, sensitive data, or internal details.
- Escalation occurs when the manipulated output drives unauthorized actions such as deleting records, approving transactions, or issuing harmful commands.
- Impact is operational and trust-related, including data exposure, fraudulent actions, and degraded confidence in GenAI outputs.
Breaches seen in the wild
- Gemini AI Breach — Google Calendar Prompt Injection: Gemini AI assistant prompt injection attack leaks sensitive Google Calendar data.
- Gemini CLI prompt injection flaw 2025: Tracebit showed a poisoned README could make Gemini CLI run hidden commands and exfiltrate developer secrets; Google fixed it in 0.1.14.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Prompt injection is a trust-boundary failure, not just a content-safety problem. The attack succeeds because systems allow untrusted text to compete with control logic inside the same prompt context. That means the core issue sits in the instruction architecture, not in user phrasing alone. Practitioners should treat prompt construction as a governance boundary, not a formatting exercise.
The control gap is prompt/data intermixing. The article’s central risk is that GenAI systems often do not preserve a hard separation between retrieved content and executable instruction. Once that boundary is blurred, the model can be persuaded to reveal secrets or act outside intended scope. The practitioner lesson is to design for separation first, then evaluate model behaviour under adversarial input.
Prompt injection becomes an identity problem when model output is allowed to trigger actions. In that case, the model is effectively acting as an intermediary decision point for data access or workflow execution. Security teams need to understand that the trust decision is no longer only “can the model answer?” but “can the model be allowed to instruct the next system?”
Layered defenses are necessary because no single control stops every injection path. The article points to prompt partitioning, rate limiting, anomaly detection, sandboxing, monitoring, and input sanitisation as complementary controls. That combination reflects the reality that direct and indirect injection exploit different parts of the stack. Practitioners should therefore govern the entire model interaction path, not just the visible prompt field.
From our research library:
- Only 23% of IT leaders were very confident in their organisation's ability to manage security and governance for GenAI deployments, according to a 2025 Gartner survey of 360 IT leaders.
- Read next: AI Agent Authorisation Guide
What this signals
Prompt injection only becomes governable when teams stop treating prompts as harmless text. Once the model can mix retrieval, instruction, and action in one flow, the programme needs explicit boundaries between content, control, and execution. That is the core design change GenAI teams should prepare for.
Prompt separation is the named control gap here: the system fails when retrieved or user-supplied content can enter the instruction plane. The practical implication is straightforward. Security teams need to map every place where content can become command and then remove that ambiguity before deployment.
AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. That trend reinforces the need to govern the full GenAI stack, not just the model endpoint.
For practitioners
- Separate instruction from content Design prompts so system instructions are not mixed with user-provided or retrieved text. Use clear partitioning and explicit handling of untrusted content before it reaches the model.
- Validate model outputs before execution Require a policy check before any model response can drive a database change, purchase, message, or other backend action. Treat output as advisory until approved by control logic.
- Harden indirect input paths Review emails, documents, web content, and other retrieval sources for embedded instructions that the model could interpret as commands. Apply filtering and sanitisation to those inputs.
- Log and monitor prompt behaviour Capture prompts, retrieval inputs, tool calls, and unusual response patterns so suspicious injection attempts can be investigated and contained quickly.
- Constrain tool-enabled assistants Limit what connected tools can do when a model is exposed to untrusted content, and isolate execution so a successful injection cannot cascade into broader system impact.
Key takeaways
- Prompt injection works because GenAI systems often fail to keep untrusted content separate from control logic.
- The risk extends beyond incorrect outputs when model responses are connected to tools, data changes, or workflow actions.
- Effective defence depends on layered containment, including prompt separation, validation before execution, monitoring, and sanitisation.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, OWASP API Security Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Prompt injection turns model interactions into privilege abuse when outputs drive actions. |
| Recommendation — Constrain agent permissions so injected instructions cannot expand identity scope or tool access. | ||
| OWASP API Security Top 10 | API6 — Unrestricted Access to Sensitive Business Flows | Injected prompts can push a model into business flows it should not control. |
| Recommendation — Gate sensitive business flows with separate approval and validation before model-triggered execution. | ||
| NIST AI RMF | GOVERN — AI Governance and Accountability | This article is fundamentally about controlling AI trust boundaries and accountability. |
| Recommendation — Define accountability for prompt handling, tool use, and output-to-action decisions in the AI governance model. | ||
| MITRE ATT&CK | TA0006;TA0040 — Credential Access; Impact | The article describes data exposure and harmful actions enabled by adversarial input. |
| Recommendation — Map prompt injection paths to credential access and impact scenarios in detection and response plans. | ||
| NIST CSF 2.0 | PR.AA-05 — Access Permissions, Entitlements and Authorizations | GenAI outputs should not bypass authorization controls for data or actions. |
| Recommendation — Apply authorization checks before any model output can access data or trigger workflow changes. | ||
Key terms
- Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
- System Prompt: The system prompt is the core instruction set that defines an LLM’s behaviour, boundaries, and response style during a session. When attackers influence or override it, they are not merely changing text. They are trying to change the model’s governing authority.
- Indirect Prompt Injection: Indirect prompt injection is an attack where malicious instructions are hidden inside content that an AI system reads later. The model may treat that content as context rather than as hostile input, which can influence tool use, data access, or workflow actions if controls are weak.
- Data Partitioning: The practice of separating one user’s or tenant’s data from another’s so that retrieval and response generation cannot cross authorised boundaries. For AI assistants, partitioning is a governance control as much as an architecture pattern, because weak partitioning turns broad access into unintended exposure.
Deepen your knowledge
NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.
Published by the NHIMG editorial team on June 12, 2026.
Updated on October 10, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org