GenAI security risks in 2026 are really identity and control gaps

At a glance

What this is: Akto argues that GenAI security risks come from control gaps across context, retrieval, tools, and action-taking, not from model quality alone.

Why it matters: This matters because IAM, PAM, and NHI teams now have to govern AI systems that can read data, use tools, and trigger downstream execution.

Context

GenAI security problems arise when a probabilistic system is allowed to operate across data, prompts, retrieval, tools, and execution at the same time. The control gap is that traditional security models assume clearer boundaries, stable inputs, and visible decision points, while GenAI systems recombine context as they run. That makes identity, authorization, and output validation part of the same governance problem, especially when agents or workflow integrations are involved.

The article is really about the failure of static control assumptions in dynamic AI environments. Once a model can read internal data, invoke tools, or influence workflows, the question changes from whether the model is accurate to whether its access, context, and actions are properly bounded. For IAM and NHI programmes, that means AI systems cannot be treated as passive consumers of data; they are active actors that need governance.

Key questions

Q: What breaks when GenAI systems are allowed to act without explicit authorization boundaries?

A: The system stops behaving like an assistant and starts behaving like an ungoverned operational actor. Without explicit authorization boundaries, a model can read, reuse, or trigger actions beyond the intent of the workflow, which turns routine prompts into business risk. The failure is not the model alone, but the absence of a clear permission model around it.

Q: Why do retrieval and context controls matter so much in GenAI security?

A: Because GenAI systems make decisions from whatever context they are given, and that context can include sensitive, manipulated, or overly broad material. If retrieval is not tightly controlled, the model may surface information or act on context that would never be acceptable in a normal access-controlled workflow. The risk is governance failure at the source, not just at the output.

Q: What signals show that an AI agent is operating outside its intended purpose?

A: Look for mismatches across identity, data, model behaviour, posture, and environment. A clean authorization trail is not enough if the agent starts touching unrelated data, follows injected instructions, drifts from its known configuration, or continues acting in a way that does not fit the task.

Q: How should organisations control sensitive data in GenAI tools?

A: Organisations should treat prompts, uploads, and model outputs as governed data flows, then apply classification, inspection, and logging at the point of use. The control objective is to stop sensitive information from entering AI workflows without visibility. That requires policy, access rules, and monitoring to work together, not as separate programmes.

Technical breakdown

Why GenAI security risk emerges across the stack

GenAI systems do not fail in one obvious layer because their behavior emerges from the interaction of instructions, retrieved context, runtime data, and downstream tools. A prompt can be harmless in isolation but unsafe once it is combined with privileged retrieval or action permissions. That is why input validation alone is insufficient. The real control problem is the end-to-end trust chain, where each layer can alter the final outcome without violating a single explicit rule. In practice, this makes control design closer to system governance than application patching.

Practical implication: Design controls around the full GenAI decision path, not around prompts or models in isolation.

How prompt injection and retrieval risks turn into control failures

Prompt injection works because the model does not naturally distinguish trusted instructions from adversarial text when both appear in the same context window. Retrieval-augmented generation creates the same issue at scale, because the model assumes retrieved content is legitimate unless the surrounding system enforces boundaries. The article’s core point is that the model itself is not the gatekeeper. Access scope, retrieval filters, and instruction hierarchy must be enforced outside the model if you want reliable control over what it can see and do.

Practical implication: Separate instruction authority from user content and restrict retrieval to tightly governed sources.

Why agentic AI changes the identity and authorization model

Once GenAI systems can call tools, update records, or trigger workflows, they behave like operational identities rather than passive interfaces. That means privilege scope, authorization boundaries, and auditability matter as much as model behavior. The article is effectively describing an identity problem: who or what is acting, under what permissions, and with what authority to move from advice to action. This is where classic IAM assumptions break down, because the actor may be software, the context may be dynamic, and the action may happen faster than manual review can intervene.

Practical implication: Treat AI agents as governed actors with explicit scopes, monitored actions, and reviewable authorization trails.

Threat narrative

Attacker objective: The attacker wants to manipulate a GenAI system into exposing data, taking unsafe actions, or amplifying trust in outputs that should not be trusted.

1. Entry occurs when adversaries use prompt injection, poisoned retrieval content, or manipulated context to influence a GenAI system’s decision path.
2. Credential or control abuse follows when the model is allowed to access data sources, APIs, or workflow tools without strong scope separation.
3. Escalation happens when unsafe outputs or tool calls are trusted and converted into downstream actions, data exposure, or unauthorized changes.
4. Impact is operational rather than purely informational, because the system can leak data, trigger fraud, or execute unintended business workflows.

Breaches seen in the wild

• Meta Muse agent hijack 2026: An undocumented Muse setting let local malware hijack Meta's personal AI agent, steal its authentication material and abuse user access.
• SalesBleed Salesforce Agentforce 2026: Three fixed Agentforce flaws let poisoned web leads make AI agents leak CRM data with zero clicks and send phishing under the agent's identity.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

GenAI security is now an identity and authorization problem, not just a model risk problem. Once a system can read internal data, invoke tools, and trigger workflows, the control question becomes who or what is acting with authority. That intersects directly with IAM and PAM because the safe use of GenAI depends on scoped permissions, auditable action paths, and clear delegation boundaries. The practical conclusion is simple: treat GenAI as an operational actor, not a chat interface.

Retrieval trust gap: the article describes a governance failure where systems assume retrieved context is valid because it was technically accessible. That assumption breaks the moment broad retrieval can surface sensitive or manipulated content into a model prompt. In practice, this means access control must exist before retrieval, not after generation. The practitioner takeaway is to govern the context source, not just the model output.

Agentic AI collapses the old separation between advice and execution. A model that can query systems or trigger workflows is no longer merely generating text. It is participating in business operations through machine identity-like behavior, which creates a new blast-radius problem when privilege is broad or poorly bounded. That is why identity governance for agentic AI must be designed around least authority, not retrospective review.

Blast-radius control is the decisive control variable for GenAI deployments. The article repeatedly shows that failures become material when one weak interaction path can affect many dependent workflows. That is the same governance pattern that appears in NHI environments, where a single over-scoped identity can propagate risk across platforms. Practitioners should read this as a mandate to narrow permissions, isolate workflows, and assume that one weak control can scale fast.

GenAI governance debt accumulates when organisations deploy before they can explain, bound, and observe behavior. The article’s strongest signal is that visibility is lagging adoption, not keeping pace with it. That means control gaps are not theoretical future issues but current programme debt. Teams should measure whether they can trace actions back to an actor, a permission, and a trusted data source before expanding deployment.

From our research library:

• Half of companies using generative AI will deploy agentic AI by 2027, according to Deloitte's 2025 Technology Predictions.
• Read next: Agentic AI Identity Maturity Model

What this signals

Agentic AI is turning identity governance into an execution-control problem. The next control boundary is not just who can authenticate, but who or what can retrieve context, call tools, and complete actions inside a workflow. That means teams should move from one-time access checks to continuous authority review across the AI action path.

GenAI programmes will fail where visibility stops at the model boundary. The risk surface now includes retrieval sources, tool chains, and downstream business systems, so the relevant question is whether the organisation can trace an action back to a permissioned source before it executes.

According to the 2026 Infrastructure Identity Survey, 67% of organisations still rely heavily on static credentials despite the risks they pose to agentic AI deployments. Static credentials remain the easiest way for AI-driven workflows to inherit more access than they should, which is why credential scope and lifecycle now sit inside the GenAI governance conversation.

For practitioners

• Define explicit authority boundaries for AI systems Map each GenAI system to the exact data sources, tools, and workflows it may access, then remove any default broad permissions that let it operate across unrelated systems.
• Separate user content from system instruction paths Use retrieval and prompt handling designs that preserve instruction hierarchy, filter untrusted inputs, and prevent user-provided text from overriding operational guardrails.
• Scope agent permissions to task-specific actions Constrain autonomous or semi-autonomous AI actions so each workflow can only perform the minimum set of calls, updates, or queries required for that task.
• Validate outputs before downstream execution Add approval, policy, or deterministic checks before AI-generated content can trigger payments, code changes, record updates, or customer-facing communications.
• Monitor retrieval and tool usage for abnormal behavior Track what sources the model retrieves, what tools it invokes, and which workflows it repeatedly touches so you can detect misuse, drift, or hidden escalation paths.

Key takeaways

• GenAI risk comes from how context, retrieval, and tool use interact, not from model quality alone.
• The biggest control failure is allowing AI systems to cross from analysis into action without explicit authorization and validation.
• Identity governance for GenAI now means constraining permissions, tracing actions, and limiting the blast radius of each workflow.

Key terms

• GenAI-Driven Cybersecurity Risk: GenAI-driven cybersecurity risk is the added exposure created when attackers use generative AI to scale, personalize, or automate malicious activity. It includes faster phishing, easier code generation, deeper impersonation, and abuse of AI systems themselves. The risk is not entirely new, but the speed and reach of attack execution are materially higher.
• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Retrieval-augmented Generation: Retrieval-augmented generation is a pattern where an AI model pulls external information before generating output. The security challenge is that access rules can weaken when data is chunked, embedded, cached, or reused, so source permissions may not automatically follow the content into the model's context.
• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions, including calling APIs, writing code, and orchestrating other agents, with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It helps security, IAM, and platform teams build the governance muscle needed for systems that act, not just respond.