Notifications
Clear all
Topic starter
21/08/2026 4:49 pm
TL;DR: AI model risk scoring becomes operationally useful only when adversarial testing, OSINT evidence, and version tracking are combined into a repeatable decision framework, according to Noma Security’s methodology. The hard part is not producing a number, but proving the score reflects real model behaviour, bounded security risk, and a governance model teams can trust.
NHIMG editorial — based on content published by Noma Security: RiskRubric.ai methodology for AI model risk assessment
By the numbers:
- The composite score uses a weighted hierarchy where Risk_Score = Σ(Pillar_Weight × Pillar_Score) and the weights sum to 100%.
Questions worth separating out
Q: What breaks when AI model risk scoring is based only on vendor claims?
A: Teams lose the ability to verify how a model behaves under attack, which means hidden jailbreak paths, data leakage, or policy failures can reach production unnoticed.
Q: Why do AI agents create a governance problem for IAM teams?
A: AI agents create a governance problem because they authenticate and act as autonomous software entities with tool access.
Q: How do security teams know if a model risk score is actually useful?
A: A useful score correlates with specific failure modes such as jailbreak susceptibility, privacy leakage, weak provenance, or inconsistent refusals.
Practitioner guidance
- Define model approval gates around evidence quality Require red team transcripts, lineage data, and OSINT checks before accepting a model score as fit for use.
- Map AI model access to identity controls Treat models that can call tools, read data, or trigger workflows as privileged non-human identities.
- Review pillar-level failures before procurement Do not rely on the final A-F grade alone.
What's in the full report
Noma Security's full article covers the operational detail this post intentionally leaves for the source:
- The full scoring methodology for converting red team and OSINT evidence into weighted risk indicators.
- Appendix-level detail on the six risk pillars and how each indicator contributes to the composite grade.
- Version tracking and continuous assessment mechanics for rescoring models after updates.
- The underlying behaviour set used by the automated red teaming engine to elicit unsafe responses.
👉 Read Noma Security's methodology for AI model risk scoring and red teaming →
AI model risk scoring: what practitioners need to validate?
Explore further
21/08/2026 5:08 pm
AI model risk scoring only works when the evidence is adversarial, not declarative. A letter grade built from live testing is more defensible than one built from vendor assurances or static checklists. The article’s methodology is strongest where it ties scores to observable model behaviour under attack conditions, which is the right direction for AI governance. Practitioners should treat any score as a starting point for control validation, not an approval stamp.
A question worth separating out:
Q: Who should be accountable when a model with a good score still causes harm?
A: The accountable owner is the team that approved the model for a specific use case and operating context. A score does not transfer responsibility. Governance should require named ownership for the model, the data it touches, and the controls that constrain its runtime behaviour.
👉 Read our full editorial: AI model risk scoring is only as strong as its test coverage
