GenAI security risks now span prompts, tools, and runtime behavior

At a glance

What this is: Akto frames GenAI security as a stack-wide problem spanning prompts, tools, data, and runtime behaviour, with agent autonomy making conventional application controls insufficient.

Why it matters: It matters because IAM, NHI, and AI governance teams now have to control what agents can access and do at runtime, not just what policies say they should do.

Context

GenAI security is a governance problem before it is a tooling problem. The control gap appears when language becomes an instruction channel, tool calls happen dynamically, and agents can act on data and systems that traditional application security assumed were separate.

That shift matters to identity teams because many GenAI deployments are already behaving like non-human identities with delegated access, short-lived decisions, and runtime tool use. Once an agent can call APIs, touch data, or trigger actions, access scope and enforcement move from design-time policy to live identity control.

Key questions

Q: What fails when GenAI systems rely on static policy instead of runtime enforcement?

A: Static policy fails because it describes intended behaviour, not live behaviour. GenAI systems can change with prompts, retrieved content, and tool availability, so the real risk appears only when the workflow runs. Without runtime enforcement, governance documents can approve a system that still leaks data, misuses tools, or acts outside scope in production.

Q: Why do over-privileged AI agents create higher business risk than ordinary users?

A: Over-privileged agents can execute many more actions than the task actually requires, so one bad decision can cascade into wrong approvals, refunds, data movement, or API calls. The risk is not just misuse. It is that the agent can act quickly, repeatedly, and without the human timing constraints that limit user mistakes.

Q: What are the signs that GenAI use is becoming unsafe in an organisation?

A: Warning signs include employees sharing sensitive information with GenAI tools, teams relying on model output without review, and inconsistent answers being used in business processes. Another signal is when security, legal, and compliance teams lack visibility into how the tools are used. At that point, the organisation has both a control gap and an audit problem.

Q: Which controls matter most when governing autonomous AI systems?

A: The most important controls are discovery, least privilege, runtime guardrails, audit logging and a clear decision owner for access changes. Together they reduce the chance that an agent can act outside approved scope and they give security teams evidence when an incident occurs.

Technical breakdown

Prompt injection turns content into command

Prompt injection happens when text that should be treated as data is interpreted as instruction by an LLM or agent. In GenAI systems, that boundary is weak because the model processes prompts, retrieved content, and tool output in the same conversational channel. Indirect injection can hide malicious instructions inside documents, emails, or web content that an agent reads during a task. The security failure is not only leakage, but misdirection: the model follows attacker-controlled instructions and can reveal context, alter decisions, or trigger tool use. The more autonomy the system has, the more damaging a single poisoned input becomes.

Practical implication: Treat untrusted content as a control boundary and filter it before it reaches model reasoning or tool execution.

Why excessive agent permissions widen blast radius

An agent with broad permissions turns a local mistake into a system-level incident. Unlike a human operator, the agent can combine access to data, internal APIs, and external tools in one continuous workflow, often without a checkpoint between actions. That means a successful injection or jailbreak does not stop at a bad answer; it can become a destructive sequence of reads, writes, and downstream actions. In identity terms, the issue is delegated authority without sufficient scoping. The agent becomes a non-human identity whose privileges are larger than the task needs, so compromise translates directly into reach.

Practical implication: Bind each agent to task-scoped authority and remove any standing access that is not needed for the current workflow.

Runtime guardrails matter more than static policy

Static policy can describe what an AI system should do, but it cannot guarantee what it will do once the workflow is live. GenAI systems change with prompts, retrieval content, model updates, and tool availability, so pre-approved rules age quickly. Runtime controls watch actual behaviour as it happens and can block risky actions, suppress leaked data, or halt execution when an agent strays outside scope. This is especially important in autonomous workflows, where the moment of compromise and the moment of impact can be the same. Governance without runtime enforcement is documentation, not protection.

Practical implication: Move enforcement to the moment of action and use live monitoring to stop unsafe tool calls or data disclosure.

Threat narrative

Attacker objective: The attacker wants to turn a trusted GenAI workflow into a path for data exposure, unauthorized action, or downstream system abuse.

1. Entry begins when attackers place malicious instructions in prompts, retrieved content, or other text that the model will read during normal operation.
2. Credential or access abuse follows when the manipulated agent uses its assigned tools, APIs, or data reach to perform actions beyond the intended request.
3. Impact occurs when the agent leaks sensitive data, performs unauthorized operations, or propagates bad instructions into connected systems.

Breaches seen in the wild

• Gemini CLI prompt injection flaw 2025: Tracebit showed a poisoned README could make Gemini CLI run hidden commands and exfiltrate developer secrets; Google fixed it in 0.1.14.
• CoPhish OAuth phishing via Copilot Studio: Datadog showed Copilot Studio agents on a Microsoft domain can front OAuth consent phishing and forward stolen tokens; no victims reported.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

GenAI security is now an identity governance problem disguised as an application security problem. Once agents can call tools, access data, and make decisions at runtime, the real control question becomes who or what is authorised to act, under which conditions, and with what blast radius. That is an NHI problem as much as an AI problem, because the agent behaves like a non-human identity with delegated authority. Practitioners should govern the agent's runtime identity, not just the model configuration.

Prompt injection exposes a verification trust gap. Security teams have long assumed that instructions and data can be separated by context, but GenAI collapses that distinction. The result is a system that can be persuaded by attacker-controlled content in ways classic application security does not model. The practical implication is that organisations need controls that verify input provenance and trust before content reaches reasoning or tool use.

Static policies cannot govern autonomous behaviour on their own. The article correctly frames runtime enforcement as the missing layer between approval and action. That mirrors a broader market shift toward live control planes for AI systems, where discovery, policy, and monitoring must operate together. The practitioner conclusion is clear: governance must move from paperwork to active enforcement if agentic AI is going to scale safely.

Exposure grows fastest where MCP-style tool chains meet overprivileged agents. The combination of dynamic tool selection, connected APIs, and broad access makes the attack surface much larger than a single model endpoint. For identity programmes, this is the point at which access control, inventory, and offboarding discipline become central to AI security. Teams should treat every connected tool as part of the agent's governance boundary.

Continuous testing is becoming a control, not just a validation exercise. The pace of jailbreaks, indirect injection, and runtime misuse means point-in-time review will always lag behind the threat. That shifts red teaming, discovery, and runtime validation from assurance activities to ongoing operating controls. The practitioner conclusion is to measure whether the system is safe while it is running, not only before it ships.

From our research library:

• 67% of organisations still rely heavily on static credentials despite the risks they pose to agentic AI deployments, according to the 2026 Infrastructure Identity Survey.
• Half of companies using generative AI will deploy agentic AI by 2027, according to Deloitte's 2025 Technology Predictions.
• Read next: Agentic AI Identity Maturity Model

What this signals

Agentic AI governance debt: organisations are adding AI capabilities faster than they are updating the control model that governs them. That gap shows up first in access scope, tool permissioning, and offboarding discipline for agents and connected services. Security teams should expect identity governance to become a live runtime function, not a periodic review cycle.

The useful operating model is no longer “approve the model, then monitor the app.” Teams need discovery, guardrails, and enforcement to stay aligned as prompts, tools, and workflows change. Where agents can reach internal APIs or sensitive data, access review alone is too slow unless it is paired with live control of action and context.

Agentic systems should be treated as governed non-human identities with measurable access boundaries. When that framing is missing, the organisation will usually notice it first through data leakage, unauthorised tool use, or runtime drift rather than through policy exceptions.

For practitioners

• Inventory every AI agent and connected tool Track each model, agent, MCP server, and API integration so security teams know what exists, what it can reach, and which systems it can influence.
• Scope agent authority to the task Assign the minimum access needed for the workflow and remove standing permissions that let an agent touch unrelated data or systems.
• Deploy runtime guardrails on tool use Block risky tool calls, sensitive data disclosure, and out-of-policy actions at execution time rather than relying on pre-approved prompts or static rules.
• Continuously test for prompt injection paths Red team indirect injection, jailbreak attempts, and poisoned retrieval content so controls are validated against the conditions that appear in production.

Key takeaways

• GenAI risk is no longer confined to prompt quality or model output. It now spans the full path from instruction handling to tool execution and runtime behaviour.
• The core weakness is the gap between policy and enforcement when autonomous systems can act before a human review step occurs.
• Discovery, task-scoped access, runtime guardrails, and continuous testing are the controls that change the risk profile, especially where agents can touch sensitive data or internal APIs.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Runtime Guardrail: A control applied while an AI agent is operating, not just during configuration or review. Guardrails can block dangerous tool calls, require approval for sensitive actions, or stop data leakage before it reaches systems or users.
• Agentic AI Identity: The complete set of credentials, permissions, and governance controls applied to an autonomous AI agent, covering authentication, authorisation, action logging, and access revocation. Distinct from traditional NHI because agent identities are often ephemeral, delegated, and multi-hop.
• MCP Server: An MCP server is a tool endpoint that connects an AI agent to external systems and data sources through Model Context Protocol. Because it extends what the agent can reach, it becomes part of the identity and access surface and must be reviewed like any other privileged connector.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It helps practitioners translate identity controls into practical governance for systems that act on their own.