Generative AI risk management needs controls beyond the model

At a glance

What this is: This is an analysis of generative AI risk management and why controls must span prompts, retrieval, tools, memory, and outputs rather than the model alone.

Why it matters: It matters because IAM, security, and governance teams need to control what GenAI systems can access, what they can do, and what evidence they leave behind.

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.

👉 Read ActiveFence's analysis of generative AI risk management for prompts, RAG, and agents

Context

Generative AI risk management starts with a simple problem: GenAI systems do not behave like fixed application logic. A chatbot, RAG workflow, copilot, or AI agent can change its response based on prompts, retrieved content, memory, tool results, and policy rules, which means traditional control points often miss the real exposure surface.

That makes this an identity and governance issue as much as a model-risk issue. When a GenAI system can see sensitive data, call tools, or operate across workflows, IAM, PAM, and lifecycle controls become part of the security design, not an afterthought.

The source article is typical of the current market problem: teams understand the concept of GenAI governance, but many still lack a practical control model that ties policy to runtime enforcement and evidence.

Key questions

Q: How should security teams govern generative AI tools that connect to core systems?

A: Treat them as non-human identities with lifecycle, access, and telemetry requirements. Assign an owner, limit privileges to the exact task, log every data flow they can trigger, and revoke access immediately when the business need ends. If a tool cannot be inventoried or monitored, it should not be connected to sensitive systems.

Q: Why do RAG systems create more governance risk than a simple chatbot?

A: RAG systems introduce an extra trust boundary because external content enters the prompt window at runtime. If access checks, source integrity, or prompt-injection defenses are weak, the model can reveal data or follow hostile instructions. The issue is not just model quality, but who can influence the retrieved context and how it is validated.

Q: What breaks when AI agents are given broad standing access?

A: Broad standing access breaks governance because the agent can move from one task to another without a fresh authorization check. That creates a control gap between intended scope and actual runtime behaviour. The result is weak accountability, limited containment, and audit trails that show activity without explaining why the activity was allowed.

Q: How do organisations know if AI governance is actually working?

A: They should be able to reconstruct a live interaction from identity context, policy outcome, accessed resources, and enforcement evidence. If the organisation can only show a policy document or a generic alert, governance is incomplete. Working AI governance leaves behind reviewable artefacts that compliance, legal, and security teams can use without guessing what happened.

Technical breakdown

Why GenAI behaves differently from fixed application logic

GenAI systems are context-sensitive systems, not deterministic rule engines. A prompt, uploaded file, retrieved document, memory entry, or tool response can alter the next output or action, which means the security boundary is not just the model endpoint. This is why prompt injection, retrieval poisoning, tool misuse, and unsafe output generation are governance problems, not just content problems. The system has to be evaluated as a full workflow that includes user input, retrieval permissions, model behavior, and downstream action paths.

Practical implication: review the entire GenAI workflow, not just the model, before approving production use.

How RAG security and prompt injection intersect with access control

Retrieval-augmented generation adds a second trust boundary because the model consumes external content at inference time. If the retrieval layer lacks permission checks or source integrity controls, the system can surface information the user should not access or treat malicious text as instruction. That makes RAG security an access-control and content-integrity problem at the same time. In identity terms, the system needs to know not only who asked, but what documents, vectors, and memory objects are allowed to shape the answer.

Practical implication: enforce document-level permissions before retrieval and treat retrieved text as untrusted context.

Why agent permissions need lifecycle and runtime governance

AI agents widen the risk because they can call tools, APIs, and workflows on behalf of a user or service process. Once an agent can move data, update records, or trigger actions, its permissions need the same lifecycle discipline as privileged service accounts and other NHIs. Static approval is not enough when the agent’s context, toolset, or memory can change after launch. Governance has to cover authorisation, logging, escalation, rollback, and post-deployment monitoring so the agent stays inside its intended boundary.

Practical implication: manage agent privileges as an NHI lifecycle problem with explicit ownership and revocation paths.

NHI Mgmt Group analysis

GenAI governance fails when organisations treat runtime context as non-security data. The article correctly shows that prompts, retrieved documents, memory, and tool outputs are all part of the attack surface. That means policy written at the procurement stage is incomplete unless it becomes enforcement in production. Practitioners should frame GenAI governance as continuous control over dynamic context, not a one-time model review.

RAG creates a governance gap because retrieval is both a permissions problem and a trust problem. If teams only secure the model, they miss the real failure mode: untrusted or overbroad content entering the prompt window. This is where identity, document access, and model behavior converge. The practical conclusion is that retrieval permissions and source integrity need to be treated as first-class controls.

AI agents are effectively non-human identities with expanding operational reach. Once an agent can call tools or act across systems, it inherits many of the governance questions once reserved for service accounts, API tokens, and privileged automation. That intersection with NHI governance is now unavoidable. Teams should classify agents, bound their privileges, and define revocation and oversight before they scale.

Dynamic context risk: this article exposes the control gap created when a system’s behavior changes after launch because prompts, retrieval, memory, and tools keep altering the trust boundary. That concept matters because traditional application security assumes a stable code path, while GenAI control failure often emerges only after deployment. Practitioners should build evidence trails that show what the system could access and do at each stage.

GenAI risk management is now an evidence problem, not just a policy problem. The article stresses monitoring, red teaming, and post-launch review because governance cannot be proven without logs, test cases, and blocked actions. Frameworks such as NIST AI RMF and OWASP only become operational when they produce audit-ready records. Teams should be prepared to demonstrate control operation, not just policy intent.

What this signals

GenAI programmes are moving from experimentation to governed operations, which means the next control gap will be proof, not policy. Teams will need evidence that retrieval permissions, output controls, and tool restrictions were actually enforced in production.

Dynamic context risk: the practical lesson for identity and security teams is that GenAI introduces a moving trust boundary. That boundary spans human prompts, machine-readable context, and non-human access paths, so lifecycle discipline becomes central to AI governance.

As agent use expands, the pressure will shift toward linking AI policy to IAM, PAM, and monitoring workflows. Programmes that cannot trace who or what accessed data, which tools were used, and what was blocked will struggle to satisfy audit and incident response demands.

For practitioners

• Inventory every GenAI workflow and owner Record each chatbot, copilot, RAG system, agent, model provider, vector store, tool integration, and business owner so no production system sits outside governance.
• Enforce retrieval permissions before prompt assembly Apply document-level access checks before content enters the model context, and treat retrieved text as untrusted input until it is authorised and validated.
• Bind agent permissions to NHI lifecycle controls Assign each agent a named owner, explicit scope, logging requirements, and revocation path, then review changes when tools, memory, or data access expand.
• Test for prompt injection and RAG poisoning before launch Red team the full workflow with hostile prompts, compromised documents, and unsafe tool paths so the evaluation covers the actual runtime boundary.
• Monitor production behaviour for drift and policy violations Track blocked prompts, risky outputs, tool calls, and access anomalies after deployment so changes in model, data, or policy do not silently widen exposure.

Key takeaways

• Generative AI risk is not confined to model outputs, because prompts, retrieval, tools, and memory all shape the attack surface.
• The strongest evidence in this article is that governance fails when runtime behaviour outpaces static policy and launch-time review.
• Practitioners should manage agents and RAG workflows with lifecycle, access, and monitoring controls that produce auditable evidence.

Key terms

• Generative AI Risk: Generative AI risk is the possibility that a model or its users will expose data, produce unsafe output, or influence decisions in ways the organisation did not intend. In practice, the risk spans confidentiality, integrity, and governance because the model can be used correctly and still create harm through misuse or over-trust.
• Retrieval-augmented Generation: Retrieval-augmented generation is a pattern where an AI model pulls external information before generating output. The security challenge is that access rules can weaken when data is chunked, embedded, cached, or reused, so source permissions may not automatically follow the content into the model's context.
• Agent-to-tool access: The permission path that lets an AI agent call a real system, query data, or trigger an operation. Unlike ordinary application access, this path can span multiple tools in one session, so the effective privilege of the server matters as much as the agent's intent.
• Dynamic context risk: The security risk created when a GenAI system’s behaviour changes as prompts, documents, memory, and tool results change. It matters because control failure may appear only after launch, when runtime context alters what the system can see, say, or do.

What's in the full article

ActiveFence's full article covers the operational detail this post intentionally leaves for the source:

• The article expands the control model for prompts, retrieval, outputs, memory, and tools across the full GenAI workflow.
• It breaks down where NIST AI RMF and OWASP map to testing, runtime guardrails, monitoring, and evidence.
• It explains how teams should think about launch review, production drift, and post-launch monitoring across GenAI use cases.
• It outlines practical risk categories for copilots, RAG systems, and agents serving regulated or customer-facing workflows.

👉 The full ActiveFence article covers the workflow controls, risk categories, and monitoring practices in more detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It helps practitioners connect access control, lifecycle discipline, and evidence to real-world identity programmes.