Join our Newsletter — 33% off our NHI Course

AI safety scoring gaps: what security teams are missing

 

(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 16497
Topic starter  

TL;DR: AI security findings can often still be scored with CVSS, but AI safety issues such as harmful outputs, unsafe tool use, and policy-violating behaviour need customer-specific, outcome-based severity models, according to INTIGRITI. The article argues that context, worst-case harm, and regulatory exposure matter more than exploit elegance when prioritising what to fix first.

Editorial analysis by NHI Mgmt Group, based on content published by INTIGRITI: “Beyond CVSS: rethinking scoring systems amidst AI Safety and Security”.

Key questions

Q: What breaks when CVSS is used alone for agentic AI systems?

A: CVSS breaks when the system can act, choose tools, and continue without human gating, because severity is no longer just a property of the code.

Q: Why does deployment context change AI safety severity?

A: The same model behaviour can have very different consequences depending on who sees it, where it appears, and what business process it affects.

Q: How do security teams know when CVSS is the wrong scoring model?

A: CVSS is the wrong model when the primary problem is harmful model behaviour rather than a technical vulnerability in the system itself.

Practitioner guidance

  • Define separate scoring tracks for security and safety findings Keep CVSS for technical vulnerabilities, but build a distinct severity table for harmful model behaviour, unsafe tool use, and policy violations.
  • Base AI safety severity on worst-case harm Score findings against the customer, audience, workflow, and the most damaging realistic outcome, including regulatory and brand exposure.
  • Capture deployment context in triage Record whether the model is internal, customer-facing, or operating inside a regulated process, because context changes the severity of the same behaviour.

Bottom line: AI safety findings do not fit cleanly inside CVSS because the main risk is often harmful behaviour, not system compromise.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 5 months ago
Posts: 16988
 

Outcome-based scoring is now a governance requirement, not a nice-to-have: AI safety findings fail when they are scored as though they were ordinary software bugs. The article is right to separate system compromise from harmful model behaviour because the downstream harm can be regulatory, reputational, or customer-facing without any classic CIA impact. Practitioners should treat severity as a business-risk classification problem, not just a vulnerability ranking exercise.

A question worth separating out:

Q: Should organisations score AI safety outcomes differently from AI security defects?

A: Yes. AI security defects are usually about compromise of the system, data, or permissions and can often be scored with CVSS. AI safety outcomes are about the harm the model or workflow produces, so organisations need a separate severity model that captures business impact, audience, and regulatory exposure.

👉 Read our full editorial: AI safety findings need outcome-based scoring, not CVSS alone



   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.