Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

AI vulnerability discovery vs execution risk: are controls keeping up?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 15051
Topic starter  

TL;DR: Security teams are increasingly able to find AI-related weaknesses, but the harder problem is execution speed, contextual abuse, and governance of agent behaviour in production, according to Terra’s analysis of the CSA Mythos brief. The control question has shifted from discovery to harness boundaries, prioritisation, and accountability before deployment becomes the incident.

NHIMG editorial — based on content published by terra: The Industry Is Fixated on AI Finding Vulnerabilities. That’s Not the Hard Problem

Questions worth separating out

Q: How should security teams govern AI models that can call tools and access data?

A: Security teams should govern AI models as non-human identities with named owners, limited scope, short-lived credentials, and continuous authorization.

Q: Why do agent frameworks create new access-risk problems for IAM teams?

A: Because they let runtime decisions, tool calls, and stateful workflows happen inside familiar development stacks, which can hide where authorisation actually occurs.

Q: What breaks when harness controls are treated like ordinary configuration?

A: The system can drift into implicit trust.

Practitioner guidance

What's in the full article

terra's full article covers the operational detail this post intentionally leaves for the source:

  • The specific argument Terra makes about the CSA Mythos brief and where it agrees or diverges from the draft priorities.
  • The Claude Code vulnerability example and the mechanics of the symbolic-link handling flaw described in the article.
  • The article's full view of the harness as a control layer, including why Terra treats it as the primary boundary for agent behaviour.
  • The prioritisation model Terra recommends for separating theoretical vulnerability from validated exposure in an AI security programme.

👉 Read terra’s analysis of AI vulnerability discovery and execution risk →

AI vulnerability discovery vs execution risk: are controls keeping up?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 3 months ago
Posts: 14635
 

Execution, not discovery, is now the limiting factor in AI security. AI programmes can surface more findings than teams can meaningfully process, but that does not reduce attacker capability. Once attackers can iterate faster than governance can respond, the real question becomes whether a weakness can be reached and weaponised. The practical conclusion for security leaders is that prioritisation and containment now matter more than counts.

A few things that frame the scale:

  • Only 19.6% of security professionals express strong confidence in their organisation's ability to securely manage non-human workload identities, according to 2024 Non-Human Identity Security Report.
  • 88.5% of organisations acknowledge that their non-human IAM practices lag behind or are merely on par with their human identity and access management efforts.

A question worth separating out:

Q: Who is accountable when a third-party AI agent misbehaves in production?

A: The organisation using the agent remains accountable for the outcomes, even if a vendor supplies the platform. Security, legal, compliance, and business owners should share responsibility for controls, monitoring, and incident response. If the agent can affect customers or regulated data, accountability cannot be outsourced with the technology.

👉 Read our full editorial: AI vulnerability discovery is no longer the hard problem



   
ReplyQuote
Share: