AI security needs separate controls for models, RAG and agents

At a glance

What this is: This is a practical AI security playbook that separates AI protection from AI-enabled SecOps and centers gateways, schemas, provenance and continuous testing.

Why it matters: It matters because IAM, NHI and broader security teams now have to govern AI call paths, tool scopes, approvals and evidence trails as access decisions move into machine-driven workflows.

By the numbers:

• When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes, and as quickly as 9 minutes in some cases.
• The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities.

👉 Read LEVO's full playbook for securing models, RAG and agentic AI

Context

AI security fails when organisations treat model protection, retrieval integrity and agent governance as a single programme. In practice, an AI system can be compromised through untrusted prompts, poisoned retrieval sources, malformed outputs or overprivileged tools, while AI used for security introduces a separate set of control, audit and accountability questions. The primary challenge is not just model risk, but the governance of every access, decision and side effect that flows through the AI stack.

That distinction matters for IAM and NHI teams because AI workflows now depend on service identities, tokens, connector scopes, human approvals and auditable execution trails. A gateway, a schema and a trace are becoming identity controls as much as application controls. LEVO’s stance is broadly typical of mature AI security programmes, but many organisations still underinvest in the operational boundary where identity, policy and runtime enforcement intersect.

Key questions

Q: How should security teams govern AI systems that can act without human approval?

A: Security teams should govern autonomous AI the same way they govern other high-risk identities, but with runtime enforcement instead of periodic review. That means tightly scoping tools, data, and actions; logging every material step; and making revocation and containment available while the session is still active. Static policy alone does not control machine-paced execution.

Q: Why do AI agents make IAM and NHI risk harder to manage?

A: AI agents can request tools, call APIs, and even create new infrastructure at machine speed, which multiplies identity events and privilege decisions. That means the control plane must handle autonomous access as a normal workload pattern, not as an exception that can wait for manual approval.

Q: What are the warning signs that an AI workflow is too risky to automate?

A: Look for free-form outputs that are parsed downstream, missing approval capture, weak source provenance, no rollback path and tool access that is broader than the task requires. Those conditions usually mean the workflow can turn a model error into a business action. If any of them are present, the workflow is not ready for unattended automation.

Q: How do organisations decide whether AI agent access reviews should be automated or manual?

A: Organisations should automate the repetitive parts of access review when AI agents create large, fast changing permission sets, then reserve human judgment for high risk exceptions. Automated reasoning helps reduce reviewer fatigue and surface outliers sooner, but it still needs policy guardrails, clear ownership, and escalation paths for privileged or sensitive access.

Technical breakdown

Gateway enforcement at the AI boundary

A gateway is the control point where every model request is checked before it reaches the model or any downstream tool. It enforces input policy, output policy, budget limits, approval requirements and trace export, which prevents untrusted text from becoming an operational instruction. For agentic workflows, the gateway also becomes the place where capability tiers and session context are enforced. Without that boundary, model behaviour is governed by prompt design alone, which is not a security control.

Practical implication: place policy enforcement at the AI boundary, not inside prompts or downstream applications.

Schema-first outputs and effectful actions

Schema-first design means the model must emit structured, typed output that downstream systems can validate before any action occurs. This is critical when the model can trigger tickets, change records, code execution or tool calls, because free-form text can be ambiguous, malformed or adversarially shaped. Deny-by-default handling ensures that malformed output is rejected instead of being interpreted by a parser, automation layer or agent framework. Typed adapters and strict validation turn model output into something auditable and bounded.

Practical implication: reject untyped or malformed model output before it can reach any effectful system.

Provenance, replay and continuous assurance

Provenance controls link data, indexes and model outputs back to signed sources, manifests and trace records. That matters because retrieval-augmented generation and agentic systems need to prove what information they saw, which sources were trusted and which actions were taken. Continuous assurance extends this into CI, adversarial test packs and weekly scorecards so that regressions are caught before release. In identity terms, this is evidence-based governance for machine decision paths, not just for human access reviews.

Practical implication: require signed sources, replayable traces and adversarial testing as release gates.

NHI Mgmt Group analysis

Security for AI and AI for Security are different control problems, not one programme. The article correctly separates AI stack protection from AI-enabled defence, because they fail differently and are governed differently. One is about preventing model, retrieval and tool compromise. The other is about using AI to accelerate analyst work without losing evidence, accountability or approval boundaries. Practitioners should separate ownership, backlogs and KPIs before they collapse both into one budget line.

Gateway policy is becoming the identity boundary for AI systems. Once models can invoke tools, the real control point is no longer the prompt, but the boundary where inputs, outputs, budgets and approvals are mediated. That boundary now intersects with IAM, PAM and NHI governance because service identities, scoped tokens and approvals determine what the model can actually do. Teams should treat the AI gateway as an access enforcement layer, not a convenience layer.

Schema-first output is the most practical way to stop AI from becoming an unsafe parser problem. Many AI failures happen when free-form text is assumed to be trustworthy enough for automation. A typed schema turns model output into a contract that can be validated, rejected and logged before side effects occur. This aligns with OWASP Agentic AI Top 10 style thinking and with NIST AI Risk Management Framework discipline. Practitioners should make structured output the default for any path that can trigger action.

Provenance is the missing control for retrievable knowledge and delegated action. Retrieval systems and agents need to show not just what they answered, but what they consumed, which source was trusted and whether the source was signed or quarantined. That is a governance gap, not just a data quality issue. The field should move toward evidence bundles for AI workloads, because replayability, takedown handling and source integrity now decide whether the system can be trusted in production.

Named concept: evidence bus for AI governance. The article points toward a model where traces, policy snapshots, evaluation results and source manifests are carried through the operating model as a single evidence layer. That concept matters because it connects security, compliance, engineering and risk on the same artefact set. Practitioners should build AI governance around evidence reuse, not separate reporting silos.

What this signals

Evidence discipline will become the operational differentiator for AI programmes. As AI moves from chat to action, the organisations that can replay sessions, prove sources and show approval history will have a materially stronger governance posture. That also means AI controls will increasingly intersect with IAM, PAM and NHI lifecycle management, because the runtime identities behind tools and connectors become part of the evidence chain.

Gateway coverage and traceability should now be treated as programme health indicators. If a model, agent or retrieval route cannot be placed behind policy enforcement, the risk is not just model misuse but uncontrolled identity delegation. For teams building mature programmes, the next step is to connect AI assurance artefacts to security operations and governance reporting rather than leaving them in engineering-only workflows.

For practitioners

• Implement a gateway on every AI route Route all model calls through a boundary that enforces input policy, output policy, budgets, approvals and trace export before any downstream action is possible.
• Make structured schemas mandatory for effectful outputs Require typed output for tool calls, automations and any path that can trigger a side effect, then reject malformed responses instead of parsing them downstream.
• Bind AI workloads to signed sources and manifests Sign corpora, indexes and retrieval manifests, attach source IDs to every answer, and quarantine or revoke data that cannot be proven.
• Run adversarial testing as a release gate Add weekly adversarial packs, CI evals and black-box or gray-box red teaming for prompt injection, poisoning and tool misuse before promotion.
• Separate Security for AI from AI for Security governance Give each programme distinct owners, budgets and KPIs, then connect them through shared evidence so operational metrics do not blur accountability.

Key takeaways

• AI risk is now an access control problem as much as a model quality problem, because agents act through identities, scopes and approvals.
• Structured outputs, provenance and replayable traces are the controls that keep AI actions auditable instead of improvisational.
• Programmes that separate AI defence from AI-enabled operations will govern faster, measure better and fail more safely.

Key terms

• AI Gateway: A control point that sits between AI applications and the models, tools, or data they call. In practice, it can authenticate requests, enforce policy, inspect runtime behaviour, and stop unsafe actions before they spread into connected systems.
• Schema-First Output: Schema-first output requires an AI system to produce structured, typed responses that can be validated before use. It reduces ambiguity, blocks malformed tool calls and prevents free-form text from being treated as executable instruction.
• Evidence Bus: An evidence bus is a structured flow of logs, policy decisions, tool calls, and model traces that supports audit and incident response. For agentic AI, it turns observability into a governance control by making decisions replayable and accountable.
• Control Provenance: The traceable origin of the evidence used to prove a control is operating. In practice, provenance matters when auditors need to know whether reports were generated independently, whether data was altered, and whether the proof can be reproduced later.

What's in the full article

LEVO's full research covers the operational detail this post intentionally leaves for the source:

• Reference blueprints for thin-wrapper LLM apps, enterprise agent gateways and private RAG deployments
• One-week rollout patterns for gateways, evals, observability and evidence bundle creation
• Concrete acceptance tests for schema pass rates, injection block rates and replay times
• Governance and compliance artefacts for AI RMF, ISO/IEC 42001 and audit-ready evidence bundles

👉 LEVO's full article covers the reference architectures, control points and release gates in more operational detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity and secrets management for practitioners building control-driven identity programmes. It gives security teams a shared vocabulary for runtime access, lifecycle enforcement and evidence-based governance.