Which frameworks should practitioners use for LLM security governance?

For AI-specific risk, use NIST AI Risk Management Framework and OWASP Agentic AI Top 10. Where credentials and workload access are involved, add OWASP NHI guidance, NIST Cybersecurity Framework, and identity controls that cover secrets, privilege, and auditability.

Why This Matters for Security Teams

LLM security governance is not just a model-quality issue. It is a control, accountability, and abuse-prevention problem that spans prompts, data, tool use, output handling, and the identity of the systems that can act on behalf of users. Practitioners often over-focus on content filters and under-specify who can call the model, what context it can see, and which downstream actions it can trigger. For a governance baseline, NIST AI Risk Management Framework is the right starting point because it frames AI risk as a lifecycle discipline rather than a point control.

The practical question is not whether an LLM can generate harmful output, but whether the organisation can evidence safe design, safe deployment, and safe operation when the model is connected to retrieval systems, APIs, and automation. That is where governance overlaps with identity and secrets management: if the model or agent can invoke tools, the access path must be owned, logged, scoped, and reviewable. In practice, many security teams encounter this only after a prompt injection or tool misuse has already occurred, rather than through intentional governance design.

How It Works in Practice

Effective governance usually combines an AI risk framework, a security control framework, and an attack-pattern view. The AI framework defines ownership, risk appetite, testing, and human oversight. The security framework defines logging, access control, change management, and incident response. The attack-pattern view helps teams test the system against realistic abuse such as prompt injection, data exfiltration, unsafe tool invocation, and jailbreak chaining. The OWASP Agentic AI Top 10 is especially useful when an LLM is allowed to plan, call tools, or operate with execution authority.

At implementation level, practitioners should map governance to concrete controls:

• Define approved use cases, prohibited use cases, and escalation paths for model failures.
• Inventory prompts, system instructions, tools, retrieval sources, and model versions.
• Restrict secrets exposure and assign workload identities so the model never receives broad standing credentials.
• Log prompts, outputs, tool calls, policy decisions, and human overrides for auditability.
• Test for model poisoning, prompt injection, and unauthorized data retrieval before release and after major changes.

Where the LLM is part of a broader application stack, NIST Cybersecurity Framework 2.0 helps structure the surrounding operational controls, while NIST AI 600-1 Generative AI Profile is useful for GenAI-specific governance overlays. These controls tend to break down when LLMs are embedded in ad hoc automation paths with unmanaged API keys, fragmented ownership, and no tested rollback process.

Common Variations and Edge Cases

Tighter governance often increases operational overhead, requiring organisations to balance model agility against review depth, especially when product teams want rapid iteration. Best practice is evolving for agentic systems, and there is no universal standard for exactly how much autonomy a production LLM may have before it becomes a separate control domain. For that reason, current guidance suggests distinguishing between read-only assistants, decision-support tools, and action-taking agents, because each tier needs different approval, logging, and access constraints.

Edge cases usually appear where the LLM touches regulated data, third-party tools, or production systems. If the model can create tickets, send emails, approve changes, or call internal APIs, governance should include identity-bound authorisation and explicit tool-level scoping, not just prompt policy. In highly connected environments, the right question is not whether the model is “safe,” but whether every action can be traced to an accountable identity, a justified permission, and a reviewable control. Where organisations extend these systems across shared platforms, CSA MAESTRO agentic AI threat modeling framework and MITRE ATLAS adversarial AI threat matrix can help teams test assumptions that policy documents alone will miss.

Related resources from NHI Mgmt Group

• How should security teams use IT governance frameworks to improve identity control?
• What frameworks should teams use when evaluating security lake governance?
• How should security teams use IAST and RASP in NHI governance?
• How should security teams use LLM-based identity risk scoring in production?