How do security teams know when CVSS is the wrong scoring model?

When CVSS Stops Being the Right Question

CVSS is designed for scoring a technical vulnerability: a flaw with a plausible exploit path, defined impact dimensions, and a severity rating that helps compare remediation work. It becomes the wrong model when the security problem is not “can this weakness be exploited?” but “what harm can this system or model produce?” That shift matters because the customer impact, not the weakness score, is what drives priority.

For teams working on AI systems, agentic workflows, or other software that can take actions, the strongest signal is whether the issue is a harmful outcome such as unsafe tool use, misleading content, unauthorized action, or policy violation. Those are operational and business-risk problems, and they need a scoring approach that reflects the worst credible consequence to the user or organisation.

What Changes When the Problem Is Behaviour, Not Vulnerability

Once the primary issue is model behaviour, the useful unit of analysis is no longer a bug report with exploitability fields. You are asking whether the system can be induced, intentionally or accidentally, to do something harmful under real operating conditions. That makes the evaluation closer to abuse-case analysis, control failure analysis, or harm modelling than conventional vulnerability severity.

In practice, this means the scoring model should answer questions like: what is the worst credible business effect, who is exposed, what trust boundary is crossed, and how repeatable is the harmful behaviour. A low technical complexity score can still hide a severe outcome if the model can trigger actions, expose data, or cause downstream decisions to be wrong.

That distinction is one reason teams should not force outcome-driven AI issues into FIRST CVSS just because they want a familiar severity number. CVSS can still describe the underlying software weakness when there is one, but it does not capture the full operational harm of a model that behaves badly inside a live workflow.

How to Decide What to Score Instead

Start by separating the technical defect from the business consequence. If there is a software flaw, unsafe configuration, or exposed service, score that issue on its own merits. If the core problem is that the model or agent can produce misleading, unsafe, or policy-breaking outcomes, then define the score around the scenario, not the code path.

A useful practitioner test is simple: if removing the technical vulnerability would not remove the risk, CVSS is probably the wrong primary model. Likewise, if the question is whether a response, recommendation, or action could cause customer harm even when no exploit exists, the scoring should be outcome-based and tied to the worst-case scenario the customer actually faces.

For vulnerability context and prioritisation, teams can still compare against a standard vulnerability source such as the NIST National Vulnerability Database, but they should not confuse record severity with real-world harm. In AI-heavy environments, the better practice is to map the behaviour to the relevant abuse pattern, then score likelihood and impact in the deployed workflow.

Practitioner Guidance

What to prioritise: Separate “system is vulnerable” from “system can cause harm.” If the latter is the real issue, anchor severity to the customer-facing consequence, not the presence of a CVE-style weakness.

What to verify: Confirm whether the risky behaviour is reproducible in production-like conditions, whether it depends on a tool, permission, or workflow step, and whether the harmful outcome persists even when the technical defect is not a classic exploit.

Decision rule: If the issue can be described only as unsafe action, misleading output, policy violation, or business loss, use a harm-based model. If you can credibly describe an exploitable technical flaw, score that separately and do not let it stand in for the business risk.

Practitioner takeaway: CVSS is useful for vulnerabilities, but it is the wrong default when the real question is how much damage a system or model can cause in the customer’s operating environment.

Related resources from NHI Mgmt Group

• What do security teams get wrong when they treat CVSS as a complete remediation decision model?
• How do security teams know when a control is built for the wrong attacker model?
• How do security teams know whether cross-model review is actually working?
• How do security teams know if their remote access model is too simple?