They show that the model can be steered by untrusted inputs into behaviours the organisation did not approve. That turns a moderation issue into a governance issue because the trust chain now includes prompts, context, tools and downstream actions. Once those paths are manipulated, accountability, safety and user trust all weaken together.
Why prompt injection becomes a governance problem
Prompt injection and jailbreaks matter because they break the assumption that model behaviour is controlled only by approved policy. Once untrusted text can steer outputs, the organisation is no longer governing a static content filter, it is governing a decision path that may include reasoning, tool use, data retrieval and action execution.
That distinction changes ownership. Security, product, legal, operations and the business function that owns the AI use case all have a stake, because the failure is not limited to bad wording. It can alter what the system decides to do, what data it touches, and what downstream action is taken in the organisation’s name.
For teams building agentic workflows, the Agentic AI Security Guide is useful because it treats inputs, memory, tools and identity as one trust chain rather than separate concerns.
How the trust chain expands beyond content moderation
Governance risk appears when the model is connected to anything that matters operationally. If the model can search, summarise, approve, draft, trigger, or hand off to another system, then a poisoned prompt is no longer just a content-quality issue. It becomes an access-and-authority issue, because the model may act with privileges the attacker does not directly hold.
This is why jailbreaks are especially important in environments that use shared sessions, delegated access or external tools. The real question is not whether the model produced unsafe text, but whether the unsafe input caused the system to cross a boundary it was supposed to enforce. That boundary may be data exposure, action approval, or business-process integrity.
When organisations want a concrete attack path, NHIMG’s Red Teaming AI Agents for Identity Abuse shows how prompt injection can become credential misuse, approval bypass and exfiltration. The Browser and Computer-Use Agent Security Guide is also relevant where the model can operate through an already signed-in browser or desktop session.
What changes in practice when the model is steerable
Once steerability is possible, the organisation has to govern more than generated content. It has to govern prompt sources, context boundaries, tool permissions, approval gates, logging, escalation paths and rollback. If any of those are weak, the model can become a route around normal controls rather than a component inside them.
The practical consequence is that accountability becomes shared but also harder to prove. If a model follows an injected instruction that came from an untrusted page, customer message, email, ticket or retrieval source, the organisation still owns the outcome. That means teams need to know which inputs are trusted, which actions are allowed, and which decisions must remain human-approved even when the model is technically capable of acting.
The strongest governance programs treat prompt injection like an integrity problem across the full operating model, not a copywriting problem. NHIMG’s Shadow AI and AI Agent Discovery Guide helps identify unmanaged AI entry points, while the AI Agent Authorisation Guide shows how to limit what an agent can do once it is in production.
Risk and Threat Considerations
Prompt injection and jailbreaks create governance risk because the attacker is exploiting delegated trust, not just misleading a classifier. If the model can pull context, call tools or influence downstream systems, the attack can convert untrusted input into unauthorised action, data leakage or process corruption.
Failure mechanism: The model accepts adversarial instructions embedded in content, then applies them through its context, tool chain or workflow integration as if they were part of legitimate operating intent.
Impact: The organisation loses control over what the AI system can safely decide, expose or execute, and the resulting failure can affect accountability, safety, auditability and user trust at the same time.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 addresses the attack surface, NIST AI RMF sets the technical controls, and ISO/IEC 42001:2023 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Prompt injection becomes governance risk when it can steer privileged agent actions. |
| ASI02 — Tool Misuse | Injected instructions often abuse agent tools and downstream actions. | |
| ASI06 — Memory & Context Poisoning | The question centers on untrusted inputs corrupting model context and decisions. | |
| Recommendation — Restrict agent permissions so untrusted prompts cannot trigger privileged actions. Constrain tool access and require approval for sensitive tool calls. Segregate untrusted context and validate retrieval before it reaches the agent. | ||
| NIST AI RMF | GOVERN-MAP | Governance risk arises because AI behavior must be managed across stakeholders and processes. |
| Recommendation — Assign accountability for AI use, oversight, and escalation before deployment. | ||
| ISO/IEC 42001:2023 | AI management system | The issue is organizational governance over AI behavior and control boundaries. |
| Recommendation — Establish AI management processes for roles, risk treatment, and change control. | ||
Practitioner Guidance
What to prioritise: Treat any AI system that can retrieve data or invoke tools as a governance-controlled workflow, not a standalone chat experience. The first control question is whether untrusted input can influence a protected action, because that determines whether the issue is merely output moderation or a true control failure.
What to verify: Confirm which sources can enter the model’s context, which tool calls are allowed without approval, and which downstream actions are reversible. If the answer cannot be traced from input to action, the system is not yet governable at the level its business owner will assume.
Practitioner takeaway: Prompt injection becomes governance risk the moment the model can use untrusted text to influence trusted decisions, privileges or actions, because the organisation then owns both the prompt path and the consequence path.
Related resources from NHI Mgmt Group
- Why do prompt injection attacks create governance risk for AI agents?
- Why do prompt injection and jailbreaking create governance risk for enterprise AI?
- Why does hidden tool-call manipulation create a higher governance risk than prompt injection alone?
- Why do non-human identities create more audit risk than human accounts?