AI Governance vs AI Security: Who Owns What in the Enterprise

Most enterprises now have an AI governance committee and a security team that both claim AI risk, and the most damaging AI failures happen in the gap between them. Governance decides whether an AI use case should exist, under what conditions and with whose accountability; security decides how the system, its data, its credentials and its supply chain are protected and monitored. This guide is an operating-model comparison: the remit of each function, the artefacts they must share, the handoff gaps where AI risk falls between governance, security, legal and data teams, and the operating model options that close them. It is for CISOs, AI governance leads, risk officers and legal and data heads.

Key takeaways

• Governance owns the decision, security owns the defence. Governance answers "should we, and on what terms"; security answers "how could this be attacked or misused, and how do we stop and detect it". Neither answer substitutes for the other.
• Shared artefacts beat shared meetings. One AI inventory, one risk tier scheme and one incident taxonomy, each with a named owner, do more than a joint committee that reviews slides.
• The gaps are predictable. Unregistered AI features in SaaS tools, agent credentials nobody approved, model updates from providers and AI incidents that are not "security" all fall between teams unless a rule assigns them.
• Exactly one accountable owner per decision. The NIST AI RMF asks for documented roles and lines of communication (GOVERN 2.1) and for executive leadership to take responsibility for AI risk decisions (GOVERN 2.3).
• The model you pick depends on size and regulation. A small firm can run AI governance from the security function; a regulated group with high-risk AI usually needs a separate governance lead with security as a standing member and veto holder on security conditions.

The short answer

AI governance is the system of decision rights, policies, risk appetite and accountability that determines which AI systems the organisation builds, buys or uses, for what purpose, and with what oversight. In NIST AI RMF terms it is the GOVERN function: policies, accountability, culture, engagement with affected parties and third-party risk. AI security is the discipline of protecting AI systems and the assets they touch against misuse, attack and failure: threat modelling, access control, secure development, supply chain assurance, monitoring and incident response. The NCSC and partner agencies' Guidelines for secure AI system development frame it across four stages: secure design, secure development, secure deployment, and secure operation and maintenance.

The practical difference: governance can approve a use case that security cannot yet protect, and security can harden a system that governance should never have approved. Each needs a gate the other respects. The neighbouring distinction between preventing harmful model behaviour and defending against adversaries is covered in the AI Safety vs AI Security Guide; this page is about organisational ownership.

Remit boundaries compared

Threat model and failure modes for each function

AI governance

• What it defends against: AI uses that are unlawful, unfair, opaque or outside risk appetite; uncontrolled spread of AI into processes; accountability that evaporates when something goes wrong.
• Architecture in words: an intake route (form or ticket) feeds an AI register; each entry gets a risk tier; the tier sets the review path (light approval or committee), required assessments and conditions; approvals expire or are re-triggered by material change; monitoring and incident outcomes feed back into the tier.
• Typical failures: a register that only captures projects the data science team builds, not AI features switched on inside existing SaaS; tiers based on the technology rather than the decision the system influences; approvals with security "conditions" nobody verifies; a committee that meets monthly while agents are deployed weekly.

AI security

• What it defends against: prompt injection, data and model poisoning, sensitive information disclosure, excessive agency, credential theft, supply chain compromise of models and packages, and abuse of AI API keys.
• Architecture in words: each registered system gets a threat model at design; identities for models, agents and tools are issued with least privilege; gateways, guardrails and logging sit on the request path; red teaming and evaluations gate release; detections feed the SOC; incidents run through the standard incident process with AI-specific playbooks.
• Typical failures: securing only the systems governance has approved, so shadow AI is invisible; treating a harmful but non-malicious output as "not our incident"; owning the agent's API key but not the decision about what the agent may do; no evaluation evidence because nobody told security a provider swapped the model version.

Each function fails where it depends on information the other holds, which is why shared artefacts matter more than any committee charter.

Shared artefacts and who owns each

Four artefacts carry the handoffs. Each needs one owner, defined contributors and a rule for what happens when the other function needs to change it.

NIST AI RMF GOVERN 1.6 asks for mechanisms to inventory AI systems resourced according to risk priorities, and NIST CSF 2.0 ID.AM-08 asks that systems, software, services and data are managed throughout their life cycles. One register satisfies both if security discovery feeds it. A practical register design is in the AI Register and AI Inventory Template.

RACI across governance, security, legal, risk and data

The table below is a decision-level summary, not a full lifecycle matrix. A fill-in version with role codes by lifecycle stage is in the AI Governance RACI Template. Here, G = AI governance lead or committee, S = security, L = legal and compliance, D = data protection or data governance, R = second-line risk, B = business system owner.

Two rows matter most. "Verify security conditions before go-live" has security as accountable because governance approvals that carry unverified conditions are the most common silent failure. "Suspend or withdraw" sits with the business owner, but governance must hold the right to require it; otherwise a committee can approve but never stop. In the EU, a deployer of a high-risk system must also assign human oversight to natural persons with the necessary competence, training and authority (EU AI Act Article 26(2)), which makes the oversight holder a named role, not a committee.

Handoff gaps where AI risk falls between teams

• AI features inside existing SaaS: a vendor switches on an assistant in a tool already approved by procurement. Governance never sees an intake; security sees no new system. Rule: any new AI capability in an approved product is a material change that re-enters intake. See the Embedded AI in SaaS Guide.
• Agent identities and delegated access: governance approves "an agent that drafts supplier emails"; the engineer grants it a mailbox scope that can send as anyone. The approval described a purpose, the credential described a capability, and nobody compared them. Rule: the approval record states permitted actions; security checks credentials against them.
• Personal OAuth grants to AI tools: an employee connects an unsanctioned AI app to corporate accounts. In the Vercel Context.ai OAuth breach, a broadly scoped OAuth grant to a third-party AI tool, with no clear owner or expiry, became the entry point.
• Silent provider model updates: a hosted model version changes behaviour. Governance's approval assumed the old evaluation results; security's tests ran on the old version. Rule: provider version changes are changes, with evaluation re-run against the tier's thresholds.
• Harm without an attacker: a chatbot gives customers wrong refund terms or a scoring model drifts into bias. Security says "no compromise"; governance has no on-call process. Rule: one incident intake for all AI incidents, routed by type. See the AI Incidents vs Security Incidents guide.
• Decommissioning: governance marks a use case retired; the API keys, service principals and vector stores remain. NIST AI RMF GOVERN 1.7 calls for processes to decommission AI systems safely, which in practice means a security checklist triggered by the governance decision.

Operating model options

Whichever model you choose, place it in the three lines structure: business owners and engineering in the first line, governance, security policy and risk challenge in the second, internal audit in the third. The Three Lines Model for Cyber Risk Guide explains the split. Committee design and terms of reference are covered in the AI Ethics Board and AI Review Committee Guide.

Decision criteria

• Regulatory exposure: if you are a provider or deployer of high-risk AI under the EU AI Act, or a regulated financial firm, choose governance-led or risk-led so non-security obligations have a dedicated owner.
• Where AI sits: if AI is mainly internal productivity tooling, security-led governance is proportionate. If AI makes or shapes decisions about customers, employees or patients, governance needs its own lead.
• Agent autonomy: once agents act with credentials (send, pay, change, deploy), security must hold a veto on permitted actions, whatever model you use.
• Speed of change: if teams ship AI features weekly, a monthly committee will be bypassed. Push low tiers to a documented fast path and reserve the committee for higher tiers.

Worked scenario: an agent for supplier payments

A finance team wants an AI agent that reads supplier invoices, matches them to purchase orders and queues payments for approval. Under a governance-led model:

1. Intake: the finance director (business owner) registers the use case. Security discovery has already flagged a trial of the vendor's agent in a sandbox tenant, so the register entry links both.
2. Tiering: governance rates impact high (financial loss, supplier relationships); security rates exposure high (reads untrusted documents, has write access to the payment queue). Combined tier: high, committee review.
3. Conditions: governance requires a named human approver for every payment and quarterly review. Security requires a dedicated agent identity, queue-only write scope with no release rights, prompt injection testing with malicious invoice content, and logging of every tool call to the SIEM.
4. Verification: before go-live, security records evidence for each condition in the register: credential scope screenshot, red team report, detection rule identifiers. Governance cannot mark the entry "live" until those fields are complete.
5. Operation: a vendor model upgrade is routed back through evaluation, and legal confirms the contract still covers its data handling.
6. Incident: an invoice carrying hidden instructions causes the agent to queue a payment to a new bank account. The human approver rejects it; security handles it as a security incident; governance reviews whether the approval threshold should tighten. Both outcomes are recorded against the same register entry.

Where identity fits

Identity is the place where governance decisions become enforceable. An approval that says "the agent may draft but not send" means nothing unless the agent's credential cannot send. The Identity Governance for AI Agents Guide covers the identity lifecycle side.

Implementing the operating model

1. Map current claims (weeks 1 to 2): list every body that believes it owns part of AI risk: security, privacy, model risk, legal, architecture boards. Record what each actually approves today.
2. Choose the model and write decision rights (weeks 2 to 4): pick one option from the table above, then write the ten or so decisions that matter with one accountable role each. Have the executive sponsor sign it, which evidences NIST AI RMF GOVERN 2.3.
3. Merge the registers (weeks 4 to 8): consolidate spreadsheets into one inventory. Feed it from security discovery: SaaS and OAuth app reviews, AI API traffic through gateways, and cloud AI service usage.
4. Agree the tier scheme and control baseline (weeks 6 to 10): combine impact and exposure criteria; map each tier to required security controls and governance conditions.
5. Wire the gates (weeks 8 to 12): make go-live depend on recorded security verification, and make material changes (new model, new tool, new data source) re-enter intake.
6. Pilot and adjust: run three real use cases of different tiers through the process, time each step and remove steps that add no decision. Keep in-flight cases on the old path rather than blocking them, so rollback is simple.

Detection and measurement

• Register coverage: AI systems found by security discovery that were already in the register, as a percentage. Below 90 percent means intake is being bypassed.
• Conditions verified: percentage of live higher-tier systems with every security condition evidenced in the register.
• Time to decision: median days from intake to approval for each tier; a rising number predicts bypass.
• Orphaned AI identities: count of agent and model credentials with no live register entry or owner.
• Change re-entry rate: provider model changes and new tool connections that went back through review, against those detected.
• Unrouted AI incidents: AI-related incidents that waited more than one working day for an owner.

Framework mapping

If you certify to ISO/IEC 42001, align its scope and roles with your ISO/IEC 27001 management system so the two do not define AI ownership twice.

Common failure patterns

• Two registers: governance keeps an approval spreadsheet and security keeps an asset list; neither is complete and they disagree about what is live.
• Conditions without verification: approvals say "subject to security review" and go live anyway because nobody owns the check.
• Security excluded from approval: the committee includes legal, ethics and business, and security hears about the agent when its first alert fires.
• No stop authority: governance can approve but has no defined way to suspend a live system, so a problematic use runs until the next review.
• Agents treated as features: the agent is approved as a product capability while its credentials are created ad hoc, outside access review. See AI agent breaches for where that has led.

Practitioner checklist

• Write down the operating model you use and the accountable role for each key AI decision; get it signed by an executive sponsor.
• Run one AI register, owned by governance and fed by security discovery of SaaS AI features, OAuth grants and AI API traffic.
• Build the risk tier from both impact and exposure criteria, and map each tier to security controls and governance conditions.
• Block go-live of higher-tier systems until security has recorded evidence for every condition.
• Record each agent's identity, owner and permitted actions in the register and check credentials against them at access review.
• Treat provider model changes, new tools and new data sources as changes that re-enter review.
• Route all AI incidents through one intake, with routing rules for security, harm and privacy types.
• Give governance a defined right to require suspension of a live system.
• Trigger a security decommissioning checklist (credentials, keys, data stores) from every governance retirement decision.
• Measure register coverage and condition verification quarterly and report both to the board committee.

Standards and references

• NIST: AI Risk Management Framework Core (GOVERN, MAP, MEASURE, MANAGE)
• European Commission AI Act Service Desk: Article 26, Obligations of deployers of high-risk AI systems
• European Commission AI Act Service Desk: Article 73, Reporting of serious incidents
• NCSC: Guidelines for secure AI system development
• NIST: Cybersecurity Framework 2.0
• NIST: SP 800-53 Rev. 5, Security and Privacy Controls
• The IIA: The Three Lines Model

Related NHI Mgmt Group resources: AI Governance Guide · AI Governance RACI Template · Security Responsibilities for AI Engineers Guide · AI Safety vs AI Security Guide