AI security needs separate controls for models, RAG and agents

At a glance

What this is: LEVO says AI security needs separate control planes for models, RAG and agents, with policy enforced at the boundary and evidence carried through the stack.

Why it matters: That matters because IAM, NHI and AI teams now have to govern what AI can see, what it can invoke and what it can change without letting those controls blur together.

Context

AI security is no longer a single control problem. Models, retrieval layers and agentic runtimes fail in different ways, so a single backlog or control owner usually leaves gaps between input policy, output handling and tool execution. The key identity issue is that AI systems now consume and emit trusted decisions, which means access, provenance and approval boundaries have to be explicit rather than assumed.

LEVO frames the problem as two distinct programmes: protecting AI systems and using AI inside security operations. That distinction matters because the governance objective changes from preventing model misuse to managing effectful behaviour, short-lived credentials, and evidence trails for autonomous or semi-autonomous actions. For practitioners, the question is not whether to secure AI, but where the control boundary must sit and which team owns it.

Key questions

Q: How should teams stop malformed AI output from causing side effects?

A: Treat AI output as untrusted until it passes a strict schema and policy check. Downstream systems should accept only typed, validated data, with deny-by-default handling for any mismatch. That prevents free-text responses, prompt injection residue or partial tool arguments from being parsed as executable instructions.

Q: Why do AI gateways matter for RAG and agent workflows?

A: Because they turn diffuse model risk into a single enforcement point. A gateway can inspect inputs, apply policy, cap budgets, require approvals and log traces before the response reaches tools or business systems. Without that control point, every integration becomes its own security exception.

Q: What breaks when AI agent provenance is not tracked?

A: You lose the ability to connect an action back to the delegator, scope, and context that authorised it. That makes incident response slower, compliance evidence weaker, and accountability unclear when an agent acts on behalf of a user or another system. Provenance is what turns activity logs into defensible governance records.

Q: How should security teams govern AI agents that inherit authority from other identities?

A: Security teams should govern AI agents by tracking identity lineage, not just credentials. That means recording the originating identity, the delegated authority path, and the runtime context for each action. If an agent can inherit permissions from humans, services, or other agents, policy has to evaluate the full chain before access is granted or continued.

Technical breakdown

Why gateways become the policy boundary for AI systems

A gateway is the enforcement point that sits between the application and the model or tool layer. In practice it can apply input filtering, output validation, schema checks, budget limits, approval requirements and trace export before any response reaches a downstream system. That matters because prompt injection, retrieval poisoning and unsafe tool calls are all boundary problems, not just model problems. Once the gateway becomes the policy choke point, security teams can treat AI traffic like other governed application flows instead of allowing each use case to invent its own guardrails.

Practical implication: Put every model and tool call through a policy gateway before it can affect production systems.

How schema-first outputs prevent untrusted text from becoming action

Schema-first design forces the model to emit structured data that downstream systems can validate before execution. This matters because free-text outputs are ambiguous, easy to misparse and unsafe when they feed scripts, tickets, code changes or operational tasks. Typed adapters, strict validation and deny-by-default handling reduce the chance that malformed or adversarial content reaches an effectful system. The technical point is simple: an AI response should be treated as untrusted until it passes machine-readable checks, not as a command because it looks plausible to a human.

Practical implication: Reject any AI output that does not match a strict schema before it can trigger side effects.

Why provenance and evidence matter for RAG and agents

Retrieval and agent systems extend the trust boundary beyond the model to include corpora, indexes, source IDs, manifests and tool traces. If those inputs are not signed, labelled and replayable, teams cannot prove what influenced a response or reconstruct why an agent acted. Provenance controls make grounding auditable, while trace export gives security, risk and compliance teams a way to replay sessions in minutes rather than guess after the fact. For agents, the control problem is not only access to tools, but whether every tool invocation can be explained and verified later.

Practical implication: Sign sources, record retrieval provenance and keep replayable traces for every AI session.

Threat narrative

Attacker objective: The attacker seeks to turn an apparently safe AI workflow into an execution path that leaks data, alters decisions or performs unintended actions.

1. Entry occurs when untrusted prompts, retrieved content or plugin inputs reach the model through an uncontrolled path.
2. Credential or policy abuse follows when the system lets those inputs influence tool selection, budget usage or approval logic.
3. Impact occurs when malformed or poisoned outputs are treated as commands and change tickets, code, data or operational workflows.

Breaches seen in the wild

• 12,000 secrets in LLM training data: Truffle Security found 11,908 live API keys and passwords hard-coded in web pages captured by Common Crawl, a dataset used to train LLMs.
• xinference PyPI compromise 2026: Three xinference releases on PyPI carried a stealer for cloud, Kubernetes and registry keys, reportedly via a compromised bot account.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI governance debt is now an operational risk, not a future control gap: organisations that treat model security, RAG security and agent security as one workstream create blind spots between policy, provenance and execution. The vendor's framing is correct to separate Security for AI from AI for Security, because ownership and success metrics differ across those programmes. Practitioners should expect audit pressure to focus on evidence, not intent.

Provenance trust gap: AI systems fail when teams cannot prove what the model saw, which sources were allowed and what changed between retrieval and action. That is especially important where RAG or agents interact with sensitive data, because the trust chain crosses data governance, IAM and operational controls. The governance lesson is that replayability is becoming a control requirement, not an after-the-fact convenience.

Schema-first control is the new safety perimeter: when outputs can trigger effects, structure is more important than eloquence. This shifts control design away from reviewing content after generation and toward blocking malformed or out-of-policy output before execution. For security teams, the practical conclusion is to treat schemas, typed adapters and deny-by-default logic as part of the control plane.

Identity boundaries now extend to AI tools and sessions: agentic systems often act through short-lived credentials, scoped tools and delegated permissions, which makes the identity of the AI runtime as important as the model itself. That intersects directly with NHI governance, because tokens, service accounts and session scopes become the operational control points. Practitioners should govern AI agents like high-risk non-human identities, not like ordinary automation.

From our research library:

• 67% of organisations still rely heavily on static credentials despite the risks they pose to agentic AI deployments, according to the 2026 Infrastructure Identity Survey.
• Only 13% of organisations feel extremely prepared for the reality of agentic AI despite the majority racing toward autonomous adoption, according to the 2026 Infrastructure Identity Survey.
• Read next: Agentic AI Security Guide

What this signals

AI governance debt: teams that combine model hardening, retrieval security and agent control into one programme often miss the evidence and identity gaps between them. The practical risk is not just unsafe generation, but uncontrolled delegation, so programme design should separate ownership while preserving a shared evidence layer.

Bounded authority becomes the differentiator for agentic systems: the organisation that can constrain tool scope, short-lived credentials and approval boundaries will scale AI faster than the one that relies on trust in model behaviour alone. That is especially true when static credentials still dominate AI deployments, according to the 2026 Infrastructure Identity Survey.

Provenance trust gap: if a team cannot replay what an AI system saw and did, it cannot defend the decision later. For practitioners, that means traceability is no longer an observability feature, it is part of the control objective for both security automation and business-facing AI.

For practitioners

• Separate Security for AI from AI for Security Create distinct backlogs, owners, budgets and KPIs for protecting AI systems versus using AI in SecOps and support workflows.
• Enforce gateway policy on every AI route Route all model, retrieval and tool traffic through a policy gateway that can apply input checks, output rules, budgets, approvals and trace export.
• Require schema validation before side effects Use schema-first outputs and typed adapters so downstream systems reject malformed or out-of-policy responses before they trigger action.
• Sign sources and preserve provenance Attach source IDs, signed manifests and retrieval evidence to corpora and indexes so you can reconstruct what influenced each answer.
• Run continuous assurance in CI and shadow traffic Add adversarial packs, weekly safety scorecards and regression gates so prompt injection, tool misuse and grounding failures block release decisions.

Key takeaways

• AI security fails when models, RAG and agents are governed as one problem, because the controls needed for input policy, provenance and tool execution are different.
• Replayable evidence and source provenance are becoming core requirements for operational trust, especially where AI decisions can trigger changes in other systems.
• Practitioners should enforce gateways, schemas and scoped authority at the boundary, then use continuous assurance to keep those controls from drifting.

Key terms

• Securing AI: Securing AI means protecting the AI system itself, including its data, prompts, tools, permissions, and outputs. For identity teams, this becomes a governance problem when the system can access secrets, call APIs, or act with delegated authority.
• AI In Cybersecurity: AI in cybersecurity is the use of machine learning and related techniques to help security teams analyse data, detect threats, and automate routine work. It is most valuable where the volume, speed, and variety of signals exceed human capacity, especially across endpoint, cloud, identity, and incident response workflows.
• Retrieval-augmented Generation: Retrieval-augmented generation is a pattern where an AI model pulls external information before generating output. The security challenge is that access rules can weaken when data is chunked, embedded, cached, or reused, so source permissions may not automatically follow the content into the model's context.
• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions, including calling APIs, writing code, and orchestrating other agents, with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity and machine identity security. It helps practitioners translate access, delegation and lifecycle controls into operational policy across modern identity programmes.