Cybersecurity assurance levels set test depth for automotive risk

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Block Harbor's blog post: Cybersecurity assurance levels in ISO/SAE 21434 fix how deeply vehicle items must be tested and reviewed without dictating which controls must be used.

Why it matters: Automotive security and supplier teams need a common assurance language so risk treatment, verification depth and independence expectations stay aligned as designs change.

Context

A cybersecurity assurance level is a governance rating, not a technical control list. It sets how much confidence a vehicle programme needs in the security work behind a component, then uses that rating to shape testing depth and reviewer independence.

In automotive supply chains, that matters because concept-phase decisions have to survive downstream design changes. Threat profiling and related assurance discussions become easier when both sides share a fixed assurance target instead of arguing over controls after implementation has started.

Key questions

Q: How is a cybersecurity assurance level determined in automotive security?

A: A cybersecurity assurance level is set from the worst credible impact of a threat scenario and the attack vector that makes the item reachable. The result is a fixed assurance target established early in the concept phase, before detailed controls are counted, so teams can plan verification depth and reviewer independence consistently.

Q: What does a higher assurance level change in vehicle cybersecurity testing?

A: A higher assurance level increases the depth, rigour, and evidence expected in vehicle cybersecurity testing. It typically means stricter test case coverage, more adversarial validation, stronger traceability to requirements, and clearer pass or fail criteria for items such as ECU interfaces, diagnostic access, boot integrity, and secure update controls.

Q: How should suppliers use assurance levels in interface agreements?

A: Suppliers should use assurance levels to fix the expected security depth for each interface, including authentication, authorisation, message integrity, diagnostic access, and update channels. The cybersecurity interface agreement should state the required assurance level, test evidence, ownership, and change control so both parties know what must be verified before integration or release.

Q: What is the difference between fuzz testing and penetration testing in automotive security?

A: Fuzz testing sends malformed, unexpected, or random inputs to find crashes, parsing flaws, and memory-safety bugs. Penetration testing is a goal-driven attack simulation that tries to exploit realistic weaknesses in systems, interfaces, and configurations. In automotive security, fuzzing is usually broader and more automated, while penetration testing is more targeted and scenario based.

Technical breakdown

How CAL is derived from impact and attack reach

A cybersecurity assurance level is determined from two inputs that are stable early in the lifecycle: the worst credible impact and the attack vector of the threat scenario. In ISO/SAE 21434's example method, the attack vector expresses how reachable the item is, from network exposure to physical access. That means a severe outcome does not automatically produce the highest level if the path to exploit it is constrained. CAL is therefore a concept-phase assurance rating, not a live reflection of implementation quality. See the Threat Profiling for Your Organisation Guide.

Practical implication: Set the assurance rating before architecture churn starts, and treat later design changes as evidence updates rather than a reason to recalculate the baseline.

How CAL changes testing depth and reviewer independence

CAL affects how much testing expands beyond routine verification. Lower levels rely on known vulnerabilities and baseline checks, while higher levels add exploratory weakness analysis, broader interaction testing, stronger fuzzing and more demanding penetration testing assumptions. The same logic applies to independence: as CAL rises, reviewers move from being simply separate people to being independent from the team or even the department that produced the work. The model is about confidence in assurance, not about the security controls themselves.

Practical implication: Match test scope and reviewer separation to the declared assurance level before release planning begins.

NHI Mgmt Group analysis

CAL is a supply-chain governance mechanism, not a control catalogue. Its value is that it gives customers and suppliers a shared assurance target before implementation choices diverge. That makes it easier to argue about evidence quality, review independence and verification depth without pretending the rating itself secures the vehicle. Practitioners should treat it as the governance layer above engineering controls, not as a substitute for them.

The real shift is from control selection to assurance allocation. ISO/SAE 21434 leaves the security control choice open, which means two teams can reach the same CAL through different technical designs. That is useful in automotive supply chains because the customer cares about demonstrated robustness, not a prescribed stack. Practitioners should use the level to align outcomes, then let engineering teams decide how to meet them.

CAL makes early lifecycle decisions persistent. Because the level is set in concept phase and stays fixed while design changes, it creates a durable reference point for verification planning. That persistence is valuable, but it also means weak assumptions made too early can echo throughout the programme. Practitioners should make the original threat scenario and impact rationale defensible, because later architecture changes will not erase a poorly chosen baseline.

Targeted Attack Feasibility turns CAL into a procurement conversation. The ISO/SAE PAS 8475 idea matters because customers often want an end-state assurance target, not just a rating assigned at the start. That gives suppliers a clearer design objective and makes interface agreements more concrete. Practitioners should expect assurance language to move further into contracts, evidence requests and supplier reviews, especially where vehicle cybersecurity is shared across tiers.

What this signals

Assurance drift is the hidden programme risk. Once a CAL is fixed, later architecture changes can create a mismatch between the original rationale and the product that actually ships. That is why automotive teams need change control that preserves the logic behind the rating, not just the label itself.

Evidence depth will matter more as vehicle supply chains mature. Shared assurance numbers only help when customers and suppliers interpret them the same way, and the PAS work suggests that consistency will become more important over time. Programme teams should expect more formal evidence requests around testing depth, reviewer independence and threat rationale.

For practitioners

• Define CAL in the concept phase Assign the assurance level from worst-case impact and attack vector before detailed design starts, then lock the rationale into the programme record.
• Use CAL to scope verification depth Expand fuzzing, penetration testing and interaction testing as the level rises, so test effort matches the confidence the programme needs.
• Separate reviewer independence from authoring teams Map higher assurance levels to independent verification and assessment roles, not just a second pair of eyes inside the same delivery team.
• Carry the CAL into supplier agreements Write the level and its rationale into the cybersecurity interface agreement so suppliers know the evidence and review burden expected of them.

Key takeaways

• Cybersecurity assurance levels are a governance tool for automotive programmes, translating threat severity and reach into a fixed assurance target.
• The level does not prescribe controls, but it does raise or lower the expected depth of testing, review independence and supplier evidence.
• Teams should set the assurance baseline early and preserve the rationale through design change, supplier coordination and verification planning.

Related NHI Mgmt Group resources: Threat Profiling for Your Organisation Guide · ISO/SAE 21434 Automotive Cybersecurity Engineering Guide · Audit, Assurance and Compliance Guide

Key terms

• Cybersecurity Assurance: Cybersecurity assurance is the confidence an organisation can justify about its security posture based on evidence, testing, and repeatable controls. It is stronger than optimism and weaker than absolute safety. In practice, it comes from visibility, validation, incident learning, and consistent reporting that supports executive decisions.
• Targeted Attack Feasibility: Targeted Attack Feasibility is the judged likelihood that a specific attacker can successfully compromise a defined asset or function. It combines the attacker’s capability, access, cost, time, and exposure of the target, and is often used to prioritise controls and testing effort in threat analysis.
• Cybersecurity Interface Agreement: A cybersecurity interface agreement is a documented security contract between parties that defines the requirements for a shared technical interface. It sets responsibilities for authentication, data handling, change management, testing, logging, and incident response so both sides know what security controls and evidence are required at the boundary.
• Threat Assessment And Risk Analysis: Threat Assessment and Risk Analysis is the process of identifying likely threats, judging their relevance, and estimating the damage they could cause. In security programs, it combines threat intelligence, asset value, control gaps, and likelihood scoring to prioritize mitigation across human IAM, NHIs, cloud, applications, and operational resilience.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security and secrets management. It helps identity and security practitioners connect governance decisions to operational controls across their programmes.