Join our Newsletter — 33% off our NHI Course

AI risk ownership and the developer accountability gap

 

(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 21826
Topic starter  

TL;DR: MIT’s Delphi study of 272 experts gives developers primary responsibility for AI security risk with full consensus, while users score far lower, highlighting a governance gap that widens as agentic systems scale, according to ZioSec’s summary of the MIT AI Risk Initiative. The deeper problem is that responsibility is being assigned to the role least structured to carry it.

Editorial analysis by NHI Mgmt Group, based on content published by ZioSec: “MIT Asked 272 Experts Who Owns AI Risk. They Picked the One Group That's Never Carried It.”.

By the numbers:

  • MIT AI Risk Initiative surveyed 272 experts on who should bear responsibility for AI risk.

Key questions

Q: What breaks when AI security responsibility is assigned only to developers?

A: Security breaks when responsibility is separated from runtime control.

Q: Why do AI users score low as owners of AI security risk?

A: AI users can initiate use, but they usually cannot govern tool choice, delegation, memory, or execution paths.

Q: How can organisations tell whether AI governance is actually working?

A: Organisations can tell AI governance is working when they can inventory every agent, explain its purpose, show who owns it, and prove that permissions are tightly scoped.

Practitioner guidance

  • Map runtime authority, not job titles Identify which team can actually constrain agent tools, credentials, and execution paths at runtime, then align ownership to that control surface.
  • Inventory every deployed agent and agent-like workflow Create a complete list of AI systems that can call tools, move data, or trigger actions without a human approval gate, including unsanctioned deployments.
  • Require continuous behavioural evidence Use live testing, logging, and attestation to prove what an agent did in production, rather than relying on developer intent or annual review cycles.

Bottom line: The article shows a governance mismatch: developers are assigned AI security responsibility, but that does not mean they control runtime behaviour.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →


This topic was modified 3 days ago by NHI Mgmt Group

   
Quote
(@mr-nhi)
Member Moderator
Joined: 5 months ago
Posts: 21634
 

Developer accountability is not the same as developer control. MIT AI Risk Initiative’s expert consensus shows where responsibility is expected to sit, but not where enforcement actually happens. In mature identity programmes, accountability only matters when the assigned actor can continuously constrain access, observe behaviour, and prove control. The practitioner conclusion is that AI governance must be built around enforceable runtime authority, not role labels.

A few things that frame the scale:

  • Only 23% of IT leaders were very confident in their organisation's ability to manage security and governance for GenAI deployments, according to a 2025 Gartner survey of 360 IT leaders.

A question worth separating out:

Q: What is the difference between responsibility on paper and control in practice for agentic AI?

A: Responsibility on paper is a governance assignment. Control in practice is the ability to limit tools, set boundaries, observe actions, and revoke authority when needed. For agentic AI, those are not the same thing, and programmes that confuse them will misplace accountability while leaving the real risk untouched.

👉 Read our full editorial: MIT's AI risk study exposes the developer accountability gap


This post was modified 3 days ago by NHI Mgmt Group

   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.