AI safety findings need outcome-based scoring, not CVSS alone

At a glance

What this is: This article argues that AI safety findings should not be ranked with CVSS alone because harmful model behaviour needs outcome-based scoring tied to business, regulatory, and user harm.

Why it matters: IAM, NHI, and AI security teams need a severity model that reflects how AI systems behave in production, because technical exploitability and real-world impact are not the same thing.

Context

CVSS is built to score technical vulnerabilities, not the broader business and behavioural harm that AI systems can create in production. That gap matters when a model produces unsafe, misleading, biased, or policy-violating outputs without any clean mapping to confidentiality, integrity, or availability.

For identity and AI governance teams, the important distinction is between a system flaw that exposes access or data and a model outcome that creates customer, regulatory, or brand harm. When AI agents and workflows are embedded in business processes, severity has to reflect the operating context, not just the exploit path.

Key questions

Q: What breaks when CVSS is used alone for agentic AI systems?

A: CVSS breaks when the system can act, choose tools, and continue without human gating, because severity is no longer just a property of the code. An agent can amplify a modest flaw into a broader operational event, so teams need a behaviour-aware layer on top of traditional scoring.

Q: Why does deployment context change AI safety severity?

A: The same model behaviour can have very different consequences depending on who sees it, where it appears, and what business process it affects. A harmful output in an internal sandbox is not equivalent to the same output in a public, regulated, or customer-facing workflow. Severity should reflect audience and exposure, not just the model flaw.

Q: How do security teams know when CVSS is the wrong scoring model?

A: CVSS is the wrong model when the primary problem is harmful model behaviour rather than a technical vulnerability in the system itself. If the issue is about unsafe tool use, misleading responses, or policy violations that create business harm, the score must be outcome-based and tied to the customer’s worst-case scenario.

Q: Should organisations score AI safety outcomes differently from AI security defects?

A: Yes. AI security defects are usually about compromise of the system, data, or permissions and can often be scored with CVSS. AI safety outcomes are about the harm the model or workflow produces, so organisations need a separate severity model that captures business impact, audience, and regulatory exposure.

Technical breakdown

Why CVSS fits AI security but not AI safety

CVSS is designed to describe vulnerabilities in systems, with exploitability and CIA impact as the core scoring logic. That works when an AI issue behaves like a conventional technical flaw, such as an unauthorized tool call or exposure of another user's data. It breaks down when the issue is model behaviour itself, because a harmful output may create material risk without any direct confidentiality, integrity, or availability failure. In those cases, the security problem is not how elegant the bypass was, but what the system caused in the real world.

Practical implication: Use CVSS for technical AI security defects, but do not force AI safety outcomes into a vulnerability model built for system compromise.

Why deployment context changes severity

AI safety severity depends on where the model runs, who sees the output, and what the output could trigger. The same harmful content can be low impact in one environment and high impact in another because customer trust, brand exposure, and regulatory sensitivity differ. Traditional environmental scoring does not fully capture those variables because it still assumes a technical vulnerability underneath the outcome. Outcome-based scoring treats the deployment context as part of the severity itself, not as an afterthought.

Practical implication: Score AI safety findings against the specific audience, workflow, and worst-case harm scenario rather than a universal severity table.

Outcome severity matters more than bypass elegance

A safety finding is about eliciting the prohibited outcome, not proving the neatest exploit chain. That means a blunt jailbreak that reliably produces harmful output can be more operationally important than a sophisticated bypass with limited real-world relevance. For AI governance, this shifts attention from exploit cleverness to repeatability, exposure, and harm category. It also aligns better with how organisations prioritise remediation when legal, customer, and reputational consequences differ from technical severity alone.

Practical implication: Prioritise findings that create reliable harmful outcomes and measurable exposure, even if the bypass looks technically unsophisticated.

Threat narrative

Attacker objective: The objective is to trigger harmful AI behaviour that creates real-world business, regulatory, or trust damage rather than to exploit a traditional system flaw.

1. Entry occurs through a technically valid prompt, tool call, or workflow interaction that does not necessarily break the underlying system controls.
2. The model or AI-enabled workflow is induced to produce unsafe, misleading, or policy-violating output rather than a classic confidentiality or integrity breach.
3. Impact emerges as customer harm, brand damage, or regulatory exposure, even when no conventional vulnerability score fully reflects the severity.

Breaches seen in the wild

• Microsoft Azure OpenAI abuse by Storm-2139: Storm-2139 used API keys leaked by Microsoft customers to hijack Azure OpenAI, bypass safety guardrails and resell access to generate harmful content.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Outcome-based scoring is now a governance requirement, not a nice-to-have: AI safety findings fail when they are scored as though they were ordinary software bugs. The article is right to separate system compromise from harmful model behaviour because the downstream harm can be regulatory, reputational, or customer-facing without any classic CIA impact. Practitioners should treat severity as a business-risk classification problem, not just a vulnerability ranking exercise.

AI safety creates a context gap that standard scoring cannot close: the same model output can be acceptable in one workflow and unacceptable in another. That makes customer-defined worst-case harm the decisive variable, especially where AI agents sit inside regulated or public-facing processes. Practitioners should make deployment context part of severity design, not part of post-triage debate.

Context-weighted severity: this is the control concept AI programmes need to operationalise. The article points to a practical truth: harmful behaviour is judged by audience, use case, and exposure, not by exploit elegance. That means AI governance teams should define severity tables that encode legal, financial, and trust impact before they scale AI into production. Practitioners should align scoring with how the system can fail in their environment, not in an abstract lab.

AI safety scoring and NHI governance intersect where agents can act, not just speak: when an AI system can call tools, trigger actions, or access back-office workflows, the boundary between model behaviour and identity-controlled execution disappears. At that point, severity must account for delegated authority, access scope, and downstream actionability, not only the textual output. Practitioners should score the harm path across both model behaviour and the permissions attached to the system.

Security teams should stop rewarding jailbreak cleverness as a proxy for impact: a lower-tech finding that consistently produces harmful outcomes can deserve higher priority than a sophisticated bypass with little operational reach. That is the right lens for AI bug bounty and internal triage alike. Practitioners should optimise for repeatable harm, not for exploit aesthetics.

What this signals

AI safety scoring is becoming a governance control point: once AI systems can affect customers, regulated workflows, or public content, severity ranking is no longer a purely technical exercise. Teams need a scoring model that tells them which issues create the highest real-world harm, not just which ones are easiest to reproduce.

The practical shift is toward dual-track triage: CVSS for system flaws, outcome-based severity for harmful AI behaviour. That split will matter more as AI agents move from advisory outputs to action-bearing workflows, because delegated execution changes the risk boundary as much as the model itself.

For practitioners

• Define separate scoring tracks for security and safety findings Keep CVSS for technical vulnerabilities, but build a distinct severity table for harmful model behaviour, unsafe tool use, and policy violations.
• Base AI safety severity on worst-case harm Score findings against the customer, audience, workflow, and the most damaging realistic outcome, including regulatory and brand exposure.
• Capture deployment context in triage Record whether the model is internal, customer-facing, or operating inside a regulated process, because context changes the severity of the same behaviour.
• Separate exploitability from outcome severity Treat a clean jailbreak, a blunt prompt, and a policy-violating response as different signals, then prioritise the one that creates the highest real-world impact.
• Review AI agent privileges alongside safety scoring If a model can call tools or trigger actions, include delegated access scope in the severity decision so behavioural risk is not assessed in isolation.

Key takeaways

• AI safety findings do not fit cleanly inside CVSS because the main risk is often harmful behaviour, not system compromise.
• Deployment context, audience, and worst-case harm are the variables that should drive severity when AI outputs create real-world impact.
• Organisations need separate scoring for AI security defects and AI safety outcomes if they want triage to reflect business and regulatory exposure.

Key terms

• AI Safety: AI safety is the discipline of preventing an AI system from taking unintended or harmful actions on its own. It focuses on the behaviour the system generates, even when no external attacker is involved. For identity teams, safety is about limiting what the agent can do once it is already operating.
• Outcome-Based Scoring: Outcome-based scoring ranks AI findings by the real-world harm they can cause, not by exploit cleverness alone. It accounts for audience, sector, regulatory exposure, repeatability, and worst-case business impact, which makes it better suited to safety issues than a pure technical metric.
• Deployment context: Deployment context is the surrounding information that determines how urgent a finding really is. It includes workload exposure, identity permissions, and data sensitivity, which together show whether a flaw is merely technical debt or a viable attack path.
• AI Security Review: An AI Security Review is a structured assessment of an AI system to find security, privacy, misuse, and governance risks before or during use. It examines model behavior, data flows, prompts, tools, access controls, logging, and failure modes, with attention to prompt injection, data leakage, unsafe actions, and policy violations.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, AI agent identity, and secrets management for practitioners who need to govern delegated access and machine action. It helps security and identity teams connect AI risk decisions to the access, privilege, and lifecycle controls their programmes already depend on.