Genai network security gaps widen as runtime controls lag

At a glance

What this is: This is an analysis of why GenAI security must move into the network and runtime layer, with the key finding that AI infrastructure is spreading faster than the controls needed to govern it.

Why it matters: It matters because IAM, NHI and security teams now have to govern prompts, APIs, agent interactions and data flows as part of the access-control surface, not as an adjacent AI problem.

Context

GenAI has moved from experimentation into business workflows, but the security model around it has not matured at the same pace. The problem is not just model safety. It is the spread of prompts, responses, tool calls and data exchange across distributed infrastructure that existing perimeter controls do not see.

This creates a governance gap for AI, identity and data security teams at the same time. When AI services, agents and APIs exchange sensitive content in normal-looking traffic, the control point shifts from the application edge to the runtime path, where inspection, policy and identity context all have to line up.

Key questions

Q: How should security teams govern AI firewalls in GenAI environments?

A: Security teams should treat AI firewalls as runtime enforcement points for prompts, outputs, and API calls, not as a complete control plane. The practical task is to combine policy checks, identity-aware access, logging, and data redaction so the model can only interact with approved users, tools, and information classes.

Q: Why do AI pilots so often become identity governance problems?

A: Because pilots frequently start with narrow intent and then accumulate extra access as teams pursue convenience or speed. Without lifecycle controls, that extra access is rarely removed. The result is governance drift, where the identity no longer matches the original use case but still retains production privileges.

Q: What are the signs that data loss controls are not keeping pace with GenAI use?

A: Common warning signs include sensitive data being pasted into prompts, users sharing regulated content with external participants, and controls that lag behind newly adopted GenAI sites. If teams cannot redact sensitive fields, classify documents in real time, or update protections quickly, users can expose business-critical information before security has a chance to intervene.

Q: What should organisations prioritise first when securing AI data exposure?

A: Prioritise discovery and classification before expanding AI use cases. If teams cannot identify sensitive unstructured data and the AI-connected paths to it, they cannot apply meaningful access control, retention, or blast-radius limits to the systems consuming that data.

Technical breakdown

Why prompt injection bypasses traditional application controls

Prompt injection works because LLMs treat natural language as both data and instruction context. A malicious prompt can ride inside a legitimate request and influence the model’s behaviour, even when the transport and authentication layers look valid. The risk is not that the network is compromised in the classic sense, but that the model receives adversarial instruction at runtime and produces unsafe output, leaks data or invokes downstream tools incorrectly. This is why API authentication alone is insufficient. Security controls have to inspect prompt content, model responses and tool messages in context, not just allow or block sessions at the edge. See the Agentic AI Security Guide.

Practical implication: Inspect prompt and response content inline, not just API access, when GenAI can trigger tools or reveal sensitive data.

How data poisoning undermines AI trust before deployment

Data poisoning targets the training or fine-tuning pipeline by corrupting the inputs that shape model behaviour. Even small amounts of tainted data can skew outputs, reduce accuracy or introduce hidden backdoors in model decision-making. The security problem is governance of source integrity, not just storage protection. If the ingestion pipeline accepts unverified sources or lacks auditability, teams may not discover the compromise until the model is already influencing business decisions. This makes model risk and supply-chain hygiene inseparable from AI security operations. See the AI Supply Chain Security and AI-BOM Guide.

Practical implication: Authenticate training sources, validate ingestion and log provenance so poisoned data can be detected before it reaches model training.

Why model theft and shadow AI are identity problems as much as network problems

Model theft often happens through weak APIs, compromised endpoints or excessive access, while shadow AI appears when employees and tools use unsanctioned services outside approved governance. Both patterns expose a deeper identity issue: who or what is allowed to access AI services, with which entitlements, and under what policy. In practice, AI workloads behave like a dense layer of non-human identities that move data, call APIs and exchange tokens continuously. Treating them as generic application traffic misses the fact that their permissions, service identities and runtime context are what make the exposure exploitable. Security teams need identity-aware controls at issuance and at runtime. See the AI Infrastructure Workload Identity Guide.

Practical implication: Govern AI services, agents and APIs as non-human identities with scoped access, monitored sessions and explicit approval boundaries.

Threat narrative

Attacker objective: The attacker wants to extract sensitive data, manipulate model behaviour or abuse connected systems through trusted GenAI interactions.

1. Entry occurs when a malicious prompt, unsafe URL or unsanctioned AI service is introduced into otherwise legitimate GenAI traffic.
2. Credential or context abuse follows when the model receives privileged prompts, tool calls or structured data that expose sensitive content or instructions.
3. Escalation happens when the same interaction path reaches connected services, plugins, APIs or agent flows that trust the model’s output too much.
4. Impact is the leakage of proprietary data, corrupted outputs, unsafe external requests or unauthorised model and API access.

Breaches seen in the wild

• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.
• LiteLLM PyPI package breach: LiteLLM PyPI supply chain attack, credentials stolen from users.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Network-aware GenAI security is now an access-control problem, not only a model-safety problem. The article is right to place enforcement in the data path because prompts, responses, tool calls and agent messages all carry security decisions in motion. That means the control plane has to understand identity, classification and context at runtime, not just perimeter reachability. Practitioners should treat GenAI traffic as governed interaction, not ordinary application throughput.

AI infrastructure creates a new form of identity sprawl that looks like workload sprawl until it fails. In GenAI environments, pipelines, copilots, APIs, agents and vector services each behave like non-human identities with their own permissions and exposure windows. When those identities are poorly inventoried, least privilege becomes theoretical and incident response becomes reactive. Practitioners should align AI governance to AI infrastructure workload identity before usage expands further.

Prompt injection is the clearest example of a runtime policy failure in GenAI systems. The weakness is not merely malicious input, but the assumption that an authenticated session implies trustworthy instructions. Once a model can reinterpret user-supplied content as operational context, classic application trust boundaries no longer hold. Practitioners should build policy enforcement around content, tool use and delegated action rather than around session acceptance alone.

Data poisoning and model theft show that AI security now spans the supply chain and the runtime path together. Training data integrity, API exposure and access scoping cannot be separated without creating blind spots. The article’s central lesson is that governance breaks when organisations treat model build-time and model run-time as different security domains. Practitioners should manage AI lifecycle risk as one continuous control surface.

Shadow AI will keep expanding until organisations can classify, authorise and monitor AI services in line with business use. Unsanctioned copilots and plugins are not just policy violations, they are uncontrolled data-processing channels. The governance gap is visibility into which services are in use and what data they can see. Practitioners should establish approved service inventory, usage policy and enforcement thresholds before adoption outpaces oversight.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Generative AI use specifically increased from 33% in 2023 to 79% in 2025, according to McKinsey’s Global Surveys on the State of AI.
• Read next: AI Infrastructure Workload Identity Guide

What this signals

AI infrastructure identity sprawl: GenAI programmes create many more governed actors than most teams initially account for, because models, agents, plugins, pipelines and vector services all need scoped access and lifecycle ownership. That is why AI governance and IAM now intersect at issuance time, not just during incident review. Practitioners should build inventory, ownership and access policy before usage scales further.

The security model for GenAI is moving toward continuous runtime enforcement, because static approval gates cannot keep pace with prompt injection, shadow AI and tool-mediated abuse. AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. Practitioners should expect the highest-risk exposures to sit around infrastructure and integrations, not only the model itself.

For practitioners

• Define GenAI as a governed runtime channel Classify prompts, responses, tool calls and agent messages as security-relevant traffic so controls can inspect and enforce policy where AI interactions actually occur.
• Inventory AI services as non-human identities Map each model endpoint, copilot, plugin, API and agent to an owner, purpose and access scope, then remove any standing access that is not tied to a business use case.
• Validate training and fine-tuning inputs Authenticate data sources, log provenance and detect anomalies in ingestion pipelines so poisoned or low-trust data is identified before model training or retraining.
• Enforce inline data-loss controls for AI flows Apply content inspection, redaction and masking to prompts and outputs so sensitive data, credentials and code cannot leave the enterprise through GenAI channels.
• Sanction or block unsupervised AI services Use service classification and usage policy to stop shadow AI from moving sensitive data into public chatbots, browser copilots and unapproved SaaS AI features.

Key takeaways

• GenAI risk is not confined to model behaviour. The wider problem is ungoverned runtime interaction across prompts, tools, APIs and shadow services.
• The article points to a control gap between rapid AI adoption and slower-maturing inspection, identity and policy enforcement in the network path.
• Practical governance now depends on classifying AI services, scoping non-human access and enforcing data controls inline before information leaves the enterprise.

Related NHI Mgmt Group resources: AI Infrastructure Workload Identity Guide · AI Supply Chain Security and AI-BOM Guide · LLM Provider API Key Security and LLMjacking Guide · Enterprise AI Copilot Security Guide

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• GenAI Runtime Defense: GenAI runtime defense is the practice of inspecting and enforcing policy on AI inputs and outputs while the application is running. It helps prevent prompt injection, jailbreaks, unsafe content, and sensitive data leakage before those interactions reach the user or downstream systems.
• AI Infrastructure Identity Sprawl: The growth of identities, tokens, keys, and delegated permissions created by AI-connected workflows across APIs, services, and cloud platforms. It becomes a governance problem when teams cannot reliably inventory who or what can act, how authority is granted, and how access is removed.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity and secrets management for teams that need to govern AI services as identities as well as applications. It helps security practitioners connect identity controls to the broader access and lifecycle issues that AI programmes create.