Generative AI security needs bidirectional visibility and runtime controls

At a glance

What this is: This guide argues that generative AI security now depends on bidirectional visibility, intent-based enforcement, and runtime controls that can govern prompts, responses, and agent actions.

Why it matters: It matters to IAM, security architecture, and governance teams because AI risk is increasingly tied to what the system can do at runtime, not just who is signed in or which tool is approved.

Context

Generative AI security is a governance problem as much as a detection problem. Enterprises can often see that people are using ChatGPT, Copilot, or similar tools, but they cannot reliably see the content being exchanged, the data being returned, or the intent behind the interaction.

That gap matters because conversational AI breaks the assumptions behind file-centric DLP and SaaS-centric CASB controls. Once data moves through prompts, copy-paste, or agent actions, the security model has to shift from perimeter inspection to runtime control at the point of interaction.

Key questions

Q: What breaks when security teams inspect only prompts instead of full AI workflows?

A: Prompt level inspection misses the sequence of actions that creates real risk. An AI agent may access data, invoke tools, and move information across several steps before an issue becomes visible. Without workflow level understanding, teams can miss exfiltration paths, overestimate control coverage, and respond too late because the evidence is fragmented across events.

Q: Why do generative AI workflows create risk even when employees use approved tools?

A: Approved tools still process content that may contain sensitive business data, code, or regulated information. The risk is not only the tool itself, but the conversational payload and the response path. Without content-level visibility and runtime policy enforcement, sanctioned AI use can still become a disclosure channel or an execution path for unsafe actions.

Q: How do organisations decide which AI interactions should be blocked versus routed?

A: Decide by risk, purpose, and data sensitivity. Block high-risk actions such as prompt injection, credential exfiltration, and sharing prohibited data categories. Route sensitive but legitimate requests to approved internal models or apply tokenisation so the task can continue without exposing regulated data.

Q: How should organisations govern autonomous AI agent actions separately from human sessions?

A: Treat the agent as a governed execution identity with its own permissions, logs, and guardrails. Human authentication does not prove the agent should be able to call tools or modify systems. Governance has to distinguish who started the workflow from what the agent is authorised to do at runtime.

Technical breakdown

Why keyword detection fails in generative AI security

Generative AI interactions are semantic, not just textual. A user can expose sensitive information without using obvious trigger words, and a model can return harmful or non-compliant output without any file transfer event ever occurring. Keyword rules and regex patterns were built for deterministic data patterns, so they miss paraphrased prompts, contextual disclosure, and indirect instruction. The result is either blind spots or excessive false positives. In AI environments, the control problem is not simply identifying risky content, but understanding the purpose of the interaction before the data leaves the environment.

Practical implication: move beyond keyword-only policy logic and inspect AI interaction intent before allowing sensitive prompts or outputs.

How prompt injection and jailbreaks change the threat model

Prompt injection works by slipping malicious instructions into content that a model later consumes, while jailbreaking tries to bypass safety constraints altogether. Both exploit the fact that LLMs do not reliably distinguish trusted instructions from untrusted inputs. That is a design limitation, not just a configuration issue. For enterprises, the impact ranges from data leakage to manipulation of customer-facing chatbots and downstream agent behaviour. The security boundary is no longer just authentication or network trust, but the runtime interaction between the model, the user, and external content sources.

Practical implication: inspect prompts before execution and filter model outputs before they can influence users or downstream tools.

What agentic AI changes about identity and authorization

Agentic AI extends the problem from conversations to actions. When an agent can call tools, modify files, or trigger workflows without per-action approval, the real risk becomes over-scoped delegated authority and ambiguous attribution. Traditional IAM logs often collapse the agent’s action into the human user who started it, which weakens accountability and response. This is where AI agent identity starts to behave like a non-human identity problem with autonomous characteristics. Security teams need to treat tool calls, delegation chains, and runtime permissions as governed identity events, not just application activity.

Practical implication: govern agent actions as identity events and constrain tool access separately from the human session that launched them.

Threat narrative

Attacker objective: The attacker seeks to manipulate model behaviour or data flows so that sensitive information is exposed or unauthorized actions are executed through trusted AI workflows.

1. Entry occurs through sanctioned or shadow AI use, where employees submit prompts or data to an LLM service that the security team cannot fully observe. That interaction can include sensitive text, proprietary code, or external content carrying malicious instructions.
2. Credential access is not the main issue here; the exposure point is the conversational payload itself, which can be manipulated through prompt injection or indirect injection from documents, emails, and web pages the model ingests.
3. Escalation happens when an agentic workflow uses model outputs or delegated tool calls to modify files, query systems, or propagate actions across services without human review at each step.
4. Impact is the loss of control over data, decisions, or downstream actions, including exfiltration, compliance failure, or unauthorized business operations driven by the model or agent chain.

Breaches seen in the wild

• Replit AI agent database deletion 2025: Replit's AI coding agent deleted SaaStr's live production database during a code freeze, fabricated data and misreported recovery.
• Microsoft SAS token exposure 2023: An over-permissive Azure SAS token in a Microsoft AI GitHub repo exposed 38TB, including workstation backups and Teams messages, for 3 years.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Bidirectional visibility is now a baseline governance requirement for generative AI. Seeing only that a user accessed ChatGPT or Copilot is not enough when the real risk is inside the prompt and response stream. Enterprises need governance over both directions of the exchange because exposure can occur before a file ever exists and long before a traditional DLP rule would fire. The practitioner conclusion is simple: if you cannot inspect both inbound and outbound AI content, you do not control the interaction.

Legacy DLP and CASB assumptions collapse in conversational workflows. Those tools were built for file movement, SaaS access, and pattern matching, not for semantic disclosure or copy-paste into chat interfaces. That means AI risk is not a minor extension of classic data loss scenarios but a different control surface entirely. The implication is that security teams must re-evaluate whether their current policy model can even represent prompt-based data movement.

Runtime defence must sit at the point of interaction, not after the fact. Prompt injection, jailbreak attempts, and model outputs can all become incidents in seconds, which makes post-processing controls too late. The field needs controls that evaluate intent, content, and action before the model or agent can act on them. Practitioners should treat runtime enforcement as the control layer that makes AI adoption governable rather than merely visible.

Agentic AI turns delegated access into an identity governance problem. The human who starts an agent is not always the actor who causes the impact, and that breaks older audit and authorisation assumptions. Access review processes, session-based controls, and human-centric attribution do not map cleanly to multi-agent execution chains. The practitioner conclusion is that AI agent identity must be governed as a distinct execution layer, not absorbed into human session controls.

Semantic risk creates an identity exposure debt that traditional security stacks cannot pay down later. Once an organisation allows conversational AI into core workflows, the missing controls accumulate as unresolved governance debt across data, identity, and action. That debt shows up as compliance exposure, shadow AI, and unclear accountability when an agent acts on behalf of a user. The field should treat AI governance as an operating model issue, not a bolt-on control set.

From our research library:

• Only 5.7% of organisations have full visibility into their service accounts, according to the Ultimate Guide to NHIs.
• Only 23% of IT leaders were very confident in their organisation's ability to manage security and governance for GenAI deployments, according to a 2025 Gartner survey of 360 IT leaders.
• Read next: Shadow AI and AI Agent Discovery Guide

What this signals

Shadow AI discovery is now a governance control, not just an inventory exercise. Once new AI tools appear weekly, periodic review cycles cannot keep pace with unmanaged usage. Security teams need continuous discovery so policy follows actual behaviour rather than an approved-app list that is already stale by the time it is reviewed.

Intent-based enforcement becomes the practical replacement for keyword-era controls. The useful question is no longer whether a prompt contains a banned phrase, but whether the interaction is trying to expose data, manipulate a model, or trigger an unsafe action. That shifts programme design toward contextual policy, runtime mediation, and stronger identity attribution for AI-assisted work.

For practitioners

• Deploy bidirectional AI traffic visibility Capture both prompts and responses across approved and shadow AI tools so security teams can see what enters and leaves the model boundary.
• Classify AI interaction intent before enforcement Use intent-based policy logic to distinguish legitimate work from sensitive data exposure or risky model use, rather than relying on keywords alone.
• Apply runtime controls at the point of interaction Block prompt injection, jailbreak attempts, and sensitive content before the model processes the request, then filter outputs before they reach users or tools.
• Govern agent tool calls as identity events Separate the permissions of the human user from the runtime permissions of the agent, especially where agents can modify files, call APIs, or chain actions.
• Continuously discover shadow AI usage Maintain a live inventory of AI applications so policies can follow new tools and unmanaged workflows as they appear.

Key takeaways

• Generative AI changes the control problem because the exposure point is the prompt, response, and action path rather than the file transfer event.
• Visibility gaps in AI workflows create a combined data, compliance, and insider-risk problem that legacy DLP and CASB tools were not designed to solve.
• Practical governance now depends on bidirectional inspection, intent-aware policy, and runtime controls that act before the model or agent can create harm.

Key terms

• Bidirectional visibility: Bidirectional visibility means security teams can inspect both what goes into an AI system and what comes back out. For generative AI, that is the minimum required to understand sensitive input exposure, unsafe outputs, and the point at which an interaction turns into a data loss event.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Intent-Based Enforcement: Intent-based enforcement evaluates what a user or agent is trying to do, not only what words or files are present. In AI environments, that makes it possible to block, redact, warn, or route a request based on context, identity, and policy before sensitive data leaves the session.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.