Why do questionnaires and security ratings fail as the main way to manage third-party risk?

Why ratings fail when dependency risk is the real problem

Questionnaires and security ratings are weak as the primary control because they flatten a live relationship into a static score. A vendor can look acceptable on paper while the service becomes more critical, the data becomes more sensitive, or a new integration quietly expands blast radius. Third-party risk is not just posture, it is operational dependence.

That is why a score should be treated as one input, not a decision rule. If the rating does not reflect how the service is actually used, how often it changes, and what data or privileges it touches, it can create false confidence and push teams toward comparison shopping instead of risk management.

One practical reason this fails is that questionnaires lag reality. By the time the answers are collected, the vendor may already have changed architecture, subcontractors, authentication paths, or support workflows. A score can therefore describe last quarter’s control environment, not today’s exposure.

What service context adds that a questionnaire cannot

Service context tells you what the rating cannot: business criticality, data sensitivity, connectivity patterns, downstream dependencies, and the consequence of failure or compromise. That is the difference between “this vendor passed review” and “this service can access production data, supports a key workflow, and would create material disruption if abused.”

This is also where evidence becomes more useful than self-attestation. Contractual claims, control questionnaires, and periodic reviews matter, but they should be anchored to observed service behavior, inventory, access paths, and monitored changes. For example, a vendor with a modest score may still deserve tighter governance if it sits on a sensitive integration chain or uses privileged tokens to reach core systems.

For readers comparing control models, the right mental shift is from vendor scoring to dependency governance. That means asking what the third party can reach, what it can change, what it can see, and what happens if its access is abused or misconfigured. NHIMG’s Ultimate Guide section on key NHI security challenges is useful here because the same visibility and overprivilege problems often show up in third-party integrations as they do in machine access more broadly.

How to use scores without letting them drive the decision

A security rating is most useful as a triage tool. It can help teams sort a large vendor population, decide where to ask for stronger evidence, and flag obvious gaps. It is not enough, by itself, to decide whether a relationship is acceptable when the service is operationally important.

The better practice is to combine three views: the supplier’s reported posture, your own service context, and live signals from monitoring and access governance. That combination is what lets you distinguish a low-risk commodity tool from a high-impact dependency with limited tolerable downtime, sensitive data exposure, or broad integration reach.

Teams should also review change velocity. A vendor that is stable and lightly integrated is easier to manage than one whose scopes, credentials, and downstream connections change frequently. A static score does not show that drift, which is why periodic revalidation alone will always trail reality.

One useful companion resource is the EU Digital Operational Resilience Act (DORA), because it frames third-party dependence as an operational resilience issue, not just a procurement check. For a similar reason, SOC 2 Trust Services Criteria can support assurance conversations, but it still needs to be paired with your own dependency and exposure assessment.

Risk and Threat Considerations

Third-party ratings can hide the conditions that make a dependency truly dangerous: privileged access, sensitive data paths, token reuse, and rapid integration sprawl. When those conditions change, the vendor score can stay the same while actual exposure rises sharply.

Failure mechanism: Questionnaire answers and point-in-time scores miss live changes in service scope, access paths, and downstream connectivity, so teams underestimate the blast radius of compromise or outage.

Impact: Organisations can approve or retain a relationship that still represents high operational and security risk, especially when the third party can touch production data, admin functions, or tightly coupled workflows.

Practitioner Guidance

What to prioritise: Classify third parties by the business service they support, the data they can access, and the access paths they hold. That classification should drive review depth, not the vendor’s overall score.

What to verify: Confirm whether the supplier’s controls still match the actual integration, not just the questionnaire response. Pay special attention to token scope, shared credentials, service accounts, privileged support channels, and any subcontracted dependency.

Decision rule: If a vendor is operationally critical or can reach sensitive systems, treat a good rating as supporting evidence only. Require service-specific evidence and monitoring before you accept the risk.

Practitioner takeaway: Ratings are useful for screening, but third-party risk is governed at the dependency level, where access, data, and change rate determine real exposure.

Related resources from NHI Mgmt Group

• How should security teams manage third-party software risk in CI/CD pipelines instead of relying only on vendor questionnaires?
• What is the difference between security ratings and traditional vendor questionnaires in third party risk management?
• How should security teams manage third-party cyber risk in practice?
• How should security teams use third-party risk questionnaires in vendor onboarding?