TL;DR: Modern AI systems create a broad threat surface spanning bias, privacy leakage, misinformation, and runtime abuse, with special emphasis on LLM failure modes such as prompt injection, jailbreaking, and training data poisoning, according to Lakera. The identity angle is clear: once AI systems can access data, tools, or credentials, governance has to extend beyond model safety into access control, secrets handling, and runtime authorization.
At a glance
What this is: Lakera argues that generative AI risk is no longer just about model behaviour, because once systems can reach data or tools the problem becomes one of identity, access, and runtime control.
Why it matters: IAM, PAM, and NHI teams need to treat AI access paths like any other privileged integration, because tool use and credential exposure can turn model issues into governance failures.
Context
AI risk is not limited to hallucinations or unsafe outputs. The governance gap appears when a generative system is allowed to access files, APIs, secrets, or downstream tools, because then model behaviour becomes an access-control problem as well as a model-safety problem. In practice, the line between AI security and identity governance disappears once the system can act on behalf of a user, service, or agent.
That shift matters for IAM and NHI programmes because AI systems often inherit privileges indirectly through the workflows they are embedded in. If an LLM can retrieve data, call tools, or surface sensitive context, then the control question is no longer only whether the model is safe. It is whether the identity, authorization, and secrets model around it is bounded, observable, and revocable.
Key questions
Q: What breaks when generative AI can access tools and data without identity controls?
A: The main failure is that model behaviour becomes an access path. If the system can retrieve data, call APIs, or use delegated credentials, then a prompt-level attack can turn into unauthorized action, data exposure, or policy bypass. The control boundary has to move from output filtering to execution authority, token scope, and revocation.
Q: Why do delegated credentials increase risk when AI agents and users are not clearly separated?
A: Delegated credentials can blur the boundary between user intent and agent autonomy. If the agent inherits the user’s identity, it may execute actions that are technically authorized but operationally unsafe, such as deleting production resources or exposing sensitive data. Clear identity separation allows additional policy checks, runtime controls, and confirmation steps before high-risk actions proceed.
Q: What are the warning signs that GenAI governance is failing in production?
A: Common signs include rising token spend without clear business growth, poor traceability of prompt flows, inconsistent routing between models, and backend services that still contain long-lived API keys. If teams cannot explain who is calling what and why, governance is not working.
Q: How should security teams govern sensitive data used by AI systems?
A: Security teams should treat AI as a data consumer that needs policy boundaries, not just authentication. Classify sensitive data, define which datasets may enter AI workflows, and monitor outputs, logs, and downstream reuse. If governance stops at login, the organisation can approve access while still losing control of the data itself.
Technical breakdown
Why generative AI becomes an identity problem
A generative system becomes an identity issue when it is no longer just producing text, but also reaching into real systems. That usually means API calls, file retrieval, workflow execution, or token use. At that point, the model sits inside an access chain: a human, service account, or agent authorizes the session; the application brokers the request; and the AI component may inherit scope that was never designed for model behaviour. The security problem is not the model alone. It is the coupling of model reasoning with permissions, secrets, and trust decisions that were built for deterministic software.
Practical implication: Separate model output safety from access authority and review every permission path the AI can trigger.
Prompt injection and jailbreaks exploit runtime trust
Prompt injection and jailbreaks are runtime attacks because they target the system’s instruction hierarchy while the model is operating. A malicious prompt can try to override policy, redirect tool use, or induce disclosure of hidden context, system prompts, or sensitive data. The key weakness is that many AI deployments treat the model as if it can safely interpret untrusted content and still remain policy-compliant. In reality, any content that reaches the model can become a control input unless the application constrains tool invocation, context exposure, and downstream actions.
Practical implication: Treat untrusted input as an active control boundary and restrict tool access to explicit policy checks.
Secrets and delegated access create the real blast radius
The biggest practical risk appears when AI systems are allowed to hold or inherit credentials. If a model, plugin, or orchestration layer can access tokens, API keys, or session cookies, then prompt abuse can become data access, workflow abuse, or account misuse. This is why identity design matters more than model tuning for many production deployments. Short-lived credentials, scoped delegation, and clear ownership determine how far a compromised AI session can move. Without those constraints, the model becomes a highly privileged intermediary rather than a bounded application component.
Practical implication: Limit delegated credentials, scope each token to a single task, and revoke access paths that the model does not need.
Threat narrative
Attacker objective: The attacker wants to turn a trusted AI workflow into an access path for data exposure, policy bypass, or unauthorized action.
- Entry occurs when an attacker sends malicious content into the model’s prompt, retrieved context, or attached document stream.
- Credential or policy abuse follows when the model is induced to reveal hidden instructions, misuse a tool, or consume delegated access beyond its intended scope.
- Impact occurs when the abused AI path exposes data, performs unauthorized actions, or widens the blast radius of a trusted workflow.
Breaches seen in the wild
- iOS apps leaking hard-coded secrets: Cybernews found 71% of 156,080 iOS apps leak hard-coded secrets, with open cloud storage and Firebase databases exposing user data.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
AI governance now has an identity layer: once a generative system can retrieve data or invoke tools, the core risk is no longer model quality alone. The governance question becomes who or what is allowed to act, under which scope, and with what revocation path. Practitioners should treat AI sessions like privileged workflows and not like ordinary content generation.
Prompt injection is an authorization failure disguised as a content problem: the attack succeeds because the application trusts untrusted text to influence tools, memory, or downstream decisions. That makes the real control point the boundary between model interpretation and execution authority. Teams should move policy enforcement to the layer that grants actions, not to the layer that merely inspects prompts.
Secrets exposure turns AI into an NHI governance problem: when tokens, API keys, or session credentials sit inside an AI workflow, the model becomes part of the machine-identity attack surface. This is where OWASP-NHI concerns and agentic AI controls converge. The practical conclusion is simple: if the AI can touch credentials, then credential lifecycle, scope, and offboarding are now AI governance issues.
Runtime authorization matters more than deployment-time assurances: the article’s core message is that AI risk is dynamic, not static. A model that is safe in test can become unsafe the moment it is connected to real data, real tools, and real privileges. Security programmes should therefore govern access at execution time and not rely on pre-deployment validation alone.
AI security is becoming part of broader identity security architecture: the most durable control model will combine model safety, access control, secrets management, and monitoring into one governance view. That does not collapse AI security into IAM, but it does mean the boundary between the two is now operational, not theoretical. Practitioners should design for bounded delegation and auditable AI action paths.
What this signals
Runtime control is the missing discipline for generative AI: model safety checks are necessary, but they do not stop a trusted workflow from doing the wrong thing once it has been handed credentials or tool access. Programmes should therefore govern the AI action path, not just the prompt path.
Agentic AI and NHI security are converging: the same questions that govern service accounts now apply to AI workflows, including ownership, scope, expiry, and offboarding. If a model can use a token, then that token needs the same lifecycle discipline as any other privileged machine identity.
AI governance debt: organisations accumulate this debt when they connect LLMs to real systems faster than they define authorization, logging, and revocation. The longer that gap persists, the more likely it is that a model issue becomes an enterprise access-control issue.
For practitioners
- Define AI execution boundaries Map exactly which files, tools, APIs, and data sources each AI workflow can reach, then remove everything not required for the task.
- Move policy checks to the action layer Block tool calls, retrieval, and delegation unless the request passes an explicit authorization decision before execution starts.
- Treat AI secrets as short-lived credentials Issue scoped, task-bound tokens for AI workflows and rotate or revoke them as soon as the session ends or changes scope.
- Separate trusted context from untrusted input Keep system instructions, private memory, and sensitive retrieval results isolated from user-controlled content wherever the model can process both.
- Log AI actions, not just prompts Capture tool invocations, retrieval events, and credential use so investigators can see what the AI actually did, not only what it received.
Key takeaways
- Generative AI risk expands sharply when models can access tools, data, or credentials, because the problem becomes one of execution authority as well as output safety.
- Prompt injection, jailbreaks, and similar runtime attacks exploit the trust boundary between untrusted input and privileged action paths.
- Governance should focus on scoped delegation, short-lived access, and auditable tool use so that AI systems cannot exceed their intended permission envelope.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, OWASP Non-Human Identity Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST AI RMF sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | The article centres on AI systems inheriting and abusing access scope through tools and credentials. |
| Recommendation — Constrain agent privileges and review every delegated access path before runtime use. | ||
| OWASP Non-Human Identity Top 10 | NHI-02 — Secret Leakage | The article explicitly warns about AI workflows exposing sensitive data and credentials. |
| NHI-05 — Overprivileged NHI | AI systems become overprivileged when they inherit broader access than the task requires. | |
| Recommendation — Scan AI workflows for exposed secrets and remove credential access from prompts and context. Reduce AI access scope to the minimum permissions needed for each task. | ||
| NIST AI RMF | GOVERN — AI Governance and Accountability | The post is fundamentally about AI governance boundaries and responsibility. |
| Recommendation — Assign clear governance owners for AI access, logging, and revocation decisions. | ||
| MITRE ATT&CK | TA0006;TA0008 — Credential Access; Lateral Movement | Prompt abuse can turn privileged AI sessions into credential access and downstream movement paths. |
| Recommendation — Map AI abuse scenarios to credential access and lateral movement detection coverage. | ||
Key terms
- Runtime Authorisation: Runtime authorisation is the practice of deciding access while a task is in progress, rather than only at provisioning time. It matters for NHIs because credentials and entitlements can change risk mid-session, especially when automation or AI agents interact with sensitive systems.
- Delegated Access: Delegated access is permission granted to one identity to act on behalf of another user, service, or system. In NHI environments, this usually appears in OAuth-connected apps and automation tooling. It is powerful, but it must be tightly scoped and reviewed because it can persist long after the original business need ends.
- Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
- Machine Identity: The digital identity of a machine, device, or workload, such as a server, container, or VM, used to authenticate it within a network. Sometimes used interchangeably with NHI, though NHI is the broader category.
Deepen your knowledge
NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It suits security and identity teams that need a practical governance baseline for machine identities, delegated access, and AI-related control design.
Published by the NHIMG editorial team on August 15, 2026.
Updated on October 12, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org