CRQ Platform Buyer's Guide: Evaluate Cyber Risk Tools

Cyber risk quantification (CRQ) platforms turn security risk into financial ranges a board can compare with other business risks, but a platform whose model you cannot inspect will produce confident numbers nobody can defend. Products range from tools that implement the Open FAIR standard for analysts, to platforms that derive loss estimates automatically from telemetry and industry loss data, to modules inside GRC suites. They differ in model transparency, data sources, how they treat controls, and how much analyst effort they need. This vendor-neutral buyer's guide helps CISOs, risk managers, finance and procurement teams define requirements, evaluate products and run a proof of concept on their own scenarios. For the method itself, including FAIR factors, calibration and Monte Carlo simulation, read the Cyber Risk Quantification Guide first.

Key takeaways

Decide the decisions first. Budget prioritisation, insurance limits, board reporting and regulatory disclosure need different outputs; buy for the decisions you will actually make.
Transparency is non-negotiable. You must be able to see every input range, distribution, data source and assumption behind a result, and change them.
Standards conformance matters. Ask whether the model follows The Open Group O-RT and O-RA standards and how that is evidenced; proprietary scores dressed as money are a warning sign.
Outside-in data is a starting point, not an answer. Industry loss data and external scans help with priors but cannot see your controls; check how internal evidence changes results.
Analysts make or break it. Every product needs trained people to scope scenarios and challenge outputs; budget for calibration training and time.

What the category does

A CRQ platform estimates how often a loss event may happen and how much it may cost, expresses the result as a distribution, and shows how controls or investments change it. The Open Group Risk Analysis Standard (O-RA, Version 2.0.1), the companion to the Risk Taxonomy Standard (O-RT, Version 3.0.1), sets out the Open FAIR method in five stages: identify the loss scenario, evaluate loss event frequency, evaluate loss magnitude, derive and articulate risk, and model the effect of controls. It expects estimates as ranges with a most likely value, calibrated to about 90 per cent confidence, and results calculated with Monte Carlo or other stochastic methods. Reports include single-number summaries (average, most likely, minimum and maximum) and distributions such as loss exceedance curves and tornado charts.

Whatever the product, check that it models the core O-RT factors explicitly rather than hiding them in a score:

Loss Event Frequency: how often a threat agent is expected to harm an asset within a timeframe, built from Threat Event Frequency (how often the threat agent acts, successful or not) and Vulnerability (the probability that a threat event becomes a loss event).
Loss Magnitude: the economic loss when a loss event occurs, expressed as a distribution rather than a single value.
Primary and Secondary Loss: direct losses to the organisation, and the further losses that follow when secondary stakeholders such as regulators, customers or litigants react.

A platform that cannot show these factors separately for a scenario cannot explain why a control investment changes the result, which is usually the question the board asks.

Platform type	How it produces numbers	Strengths	Limits to probe
Analyst-led Open FAIR tools	Analysts scope scenarios and enter calibrated ranges; the tool runs simulations	Transparent, defensible, close to the standard	Analyst time; quality depends on calibration
Data-driven or automated platforms	Combine industry loss and incident data, firmographics and telemetry from security tools to estimate frequency and magnitude	Fast coverage of many scenarios; less manual effort	Black-box risk; how internal control evidence is weighted
Outside-in rating based platforms	Use external scan data and company size to infer exposure and loss	Quick view of third parties and portfolios	Cannot see internal controls; weak for internal decisions
GRC suite CRQ modules	Add quantification to existing risk registers	One system for risks, controls and evidence	Often shallow models; check simulation depth
Insurer and broker models	Estimate loss for insurance limit and retention decisions	Aligned to insurance placement	Built for underwriting, not security investment

When you need it

Prioritising investment: you need to compare security projects by risk reduced per pound or dollar spent, rather than by heat map colour.
Board and ERM reporting: NIST IR 8286B (February 2025) describes adding risk priorities and responses to the cybersecurity risk register so they feed an enterprise risk register and a composite view of risk. NIST CSF 2.0 GV.RM-06 asks for a standardised method for calculating and prioritising cyber risks. See the NIST IR 8286 Cybersecurity and ERM Integration Guide.
Insurance decisions: setting limits and retentions against plausible loss ranges. See the Cyber Insurance Guide.
Disclosure and materiality support: US-listed companies under the SEC's 2023 cybersecurity disclosure rules must describe their processes for assessing and managing material cyber risks in annual reports and generally disclose material incidents on Form 8-K within four business days of determining materiality. A CRQ model can inform, but never replace, the legal materiality judgement. Check the rules' current status with counsel.
Risk appetite: expressing tolerance as financial thresholds that can be tested. See the Risk Appetite and Tolerance for Cyber Guide.

Decision guidance: a smaller organisation with one or two trained analysts can start with spreadsheet-based Open FAIR models for its top five scenarios, and buy a platform only when the number of scenarios or the reporting cadence outgrows the spreadsheets. Large or regulated organisations that need repeatable analysis across business units, audit trails and integration with GRC and security telemetry usually benefit from a platform. Organisations whose main need is third-party portfolio risk may prefer outside-in tools, accepting their limits.

Define requirements

Decisions and audience: list the decisions (budget, insurance, board, disclosure, third party) and who consumes the output (board, CFO, CRO, CISO).
Scenario scope: the first 10 to 20 loss scenarios, defined by asset, threat community, effect (confidentiality, integrity, availability) and loss type. Ransomware on core systems, identity compromise of a privileged administrator and theft of customer data via a stolen API key are common starters.
Method: whether you require Open FAIR conformance, which control model (for example FAIR-CAM) you want, and whether you need sensitivity analysis.
Data: which internal sources (incident history, control test results, vulnerability and identity posture data, business impact analyses, finance data) and which external sources (industry loss data, threat intelligence) the platform should use.
Outputs: loss exceedance curves, annualised loss ranges, return on control investment, risk against appetite, and board-ready reports.
Governance: review and approval workflow for analyses, version history and audit trail, mapped to NIST CSF 2.0 GV.RM-02, GV.RM-03 and ID.RA-05 and SP 800-53 RA-3 and PM-9.
People: how many analysts will use it, and the training they need. See the Open FAIR Foundation Certification Guide.

Where identity fits

Many of the largest loss scenarios start with identity: stolen administrator credentials, help desk resets, leaked API keys and over-privileged service accounts. Check whether the platform can take identity posture data as control evidence (MFA coverage, privileged account counts, secrets rotation, NHI ownership) and show how improving it changes loss event frequency. If identity controls are invisible to the model, investments in them will look worthless. See the Identity Controls for Cyber Insurance Questionnaires Guide.

Evaluation criteria

Criterion	What good looks like	How to test it
Model transparency	Every factor, range, distribution and data source visible and editable; documented algorithms	Trace one result back to all its inputs during the trial
Standards alignment	Uses O-RT factor definitions and O-RA method; states conformance evidence	Compare factor names and definitions with O-RT
Calibration support	Ranges with most likely values, confidence, and guidance for subject matter experts	Have an expert enter estimates and review the workflow
Simulation and outputs	Monte Carlo with configurable iterations, loss exceedance curves, percentiles, tornado charts	Rerun a scenario and check stability of results
Control modelling	Controls affect specific factors (frequency, vulnerability, magnitude) with evidence, not a flat discount	Model one control change and explain the effect
Data provenance	Named external data sets with dates, and clear weighting of internal evidence	Ask for the source and age of the industry data used in a result
Integration	Imports from GRC, vulnerability, identity and security tools through APIs; exports to BI and GRC	Import one control data set and see the change
Governance and audit trail	Versioning, approvals, change logs and reproducible results	Restore last month's analysis and reproduce its numbers
Reporting	Board, executive and analyst views with plain explanations of uncertainty	Ask your CFO to read a sample report
Security and data handling	Role-based access, MFA, data residency options, read-only connectors	Review where scenario and finance data is stored

Questions to ask vendors

Show us every input behind this result. Can we change any of them, and does the result update reproducibly?
Which standard does your model follow? How do your factor definitions map to The Open Group O-RT?
Which external loss and incident data sets do you use, how old are they, and how relevant are they to our sector and size?
How do our own control test results and telemetry change the estimates? What happens if a data feed stops?
How do you model a control: as a change to specific factors, or as a percentage discount?
How do you show uncertainty to a board without implying false precision?
What effort does a typical customer need for the first ten scenarios, and how many trained analysts do they have?
Can results be exported with full assumptions for audit, insurers or regulators?
What permissions do your connectors need into our security and GRC tools?
Who validates your model, and can we see the validation or back-testing results?

Red flags

A single precise loss figure with no range or distribution.
Inputs or weights hidden as proprietary, so results cannot be challenged or reproduced.
Scores built from ordinal ratings and then multiplied into money; O-RA cautions against relying on ordinal scales.
Results that do not change when you improve a major control.
Industry data presented without source, date or sector relevance.
Claims that the platform determines materiality or replaces risk analysts.
Connectors that need write access to security tools.

Proof of concept

Pick three scenarios: one you have already analysed manually (as a benchmark), one identity-led scenario such as compromise of a privileged administrator account, and one availability scenario such as ransomware on a core platform.
Gather inputs: incident history, control evidence, business impact data and finance figures for each, and run calibration training for the subject matter experts who will estimate.
Run each product: build the scenarios with each vendor's team present, and record analyst hours.
Trace and challenge: for each result, trace every input to its source, change two key assumptions and confirm the output moves as expected.
Test a decision: model a proposed investment (for example phishing-resistant MFA for administrators) and compare the projected reduction in loss with the cost.
Test reporting: give the board-level output to a finance executive and ask what decision it supports and what they do not trust.
Test governance: version an analysis, change it, and reproduce the earlier result.
Score: rate transparency, conformance, data relevance, effort, decision usefulness and cost.

Deployment and integration

Start small: top 10 scenarios agreed with the risk committee, then expand by business unit.
Connect to the risk register: link each quantified scenario to the enterprise risk register entries and risk owners, following the NIST IR 8286 approach. See the Risk Register Template.
Feed control evidence automatically: import control test results and posture data on a schedule, and log when an analysis used stale evidence.
Review results with finance: agree loss magnitude categories and figures with finance so the numbers are trusted.
Report consistently: use the same scenario set and format each quarter so trends are meaningful. See the Board Cyber Risk Reporting Guide.

Common deployment pitfalls include quantifying everything at once, letting vendor default data replace your own evidence, presenting medians without ranges, and leaving no trained analyst to challenge the output. Measure the programme with: number of decisions supported by quantified analysis each quarter, percentage of top risks with a current analysis (under 12 months), share of inputs backed by internal evidence, analyst hours per scenario, and the difference between forecast and actual losses where incidents occur.

Standards and references

How NHI Mgmt Group can help

We provide independent requirements, RFP and evaluation support. Browse vendors in our products directory or contact us.

Cyber Risk Quantification Platform Buyers Guide: Model Transparency, Data and a Proof of Concept