What are the signs that a Common Criteria evaluation is likely to fail late?

What late-failure signs mean in a Common Criteria evaluation

Late failure usually means the evaluation has moved past paper quality and into evidence scrutiny, where mismatches become expensive to repair. The most reliable early warning is a gap between what the Security Target claims and what the delivered product actually contains, especially when configuration, crypto, or platform scope has shifted since the draft was written. That gap tends to become visible only once the evaluator begins tracing evidence end to end.

A second signal is that the assurance story is not closed under change. If patching, dependency updates, build options, or library substitutions are happening faster than the certification baseline is being maintained, assessors will eventually find that the evidence no longer describes the product under review. At that point, the issue is not a missing document, but a broken chain of trust between design, build, and delivered configuration.

The third sign is weak proof around the cryptographic boundary and related supporting assumptions. When the boundary is unclear, when validated algorithms are referenced without current validation status, or when the evaluation relies on “intended” configuration rather than reproducible configuration, the review often runs smoothly until the assessor asks for exact artefacts. That is where late-stage rework appears.

Where late-failure risk accumulates in the evidence package

Late failure is rarely caused by one dramatic defect. It is usually the accumulation of small inconsistencies: a Security Target that describes one product revision, test evidence that reflects another, or design documentation that omits a dependency now inside scope. Those inconsistencies are survivable early, but they become blocking when the assessor needs one coherent assurance chain.

Configuration control is often the decisive pressure point. If the product can be rebuilt in more than one way, or if the certification baseline is not the same artifact the team is shipping, the evaluator cannot safely rely on the evidence set. Likewise, if third-party libraries, patches, or cryptographic components changed after the evidence was frozen, the evaluation can fail not because the product is weak, but because the assurance case is no longer provable.

Algorithm validation is another common trigger. When the evaluation depends on cryptographic claims, the team needs to show that the algorithms, modules, and operating modes are still the ones covered by the validation story. If that linkage is stale, missing, or ambiguous, the failure tends to surface late because the issue only becomes obvious when the evaluator checks the exact boundary and configuration state.

What practitioners should watch before the assessor does

Three practical checks usually separate a recoverable review from a late collapse. First, verify that the shipped build, the Security Target, and the evidence package all describe the same thing. Second, verify that every change since the baseline is either absorbed into the certification story or explicitly excluded. Third, verify that cryptographic components, dependencies, and platform assumptions are still aligned with the claimed assurance scope.

It also helps to treat the evaluation package as a living control surface rather than a final binder. If engineering teams are still changing libraries, toggling build flags, or adjusting deployment topology while the evidence is being prepared, the certification team needs a formal point of control. Without that, the assessor discovers drift first, and late discovery almost always means schedule impact.

What to verify: Confirm that the baseline is reproducible from source, build, and deployment artefacts, and that the exact configuration under test is the one described in the Security Target. If that cannot be proven quickly, assume the evaluation is already at risk.

Decision rule: If a patch, library upgrade, or crypto change alters the certified boundary, treat it as a certification-impacting change until the evidence package is updated and rechecked. Do not wait for the assessor to raise the mismatch.

Common mistake: Teams often assume that strong functional test results can compensate for weak configuration proof. In common criteria, they cannot. The evaluation can still fail if the assurance chain does not show control, continuity, and exact product identity.

Practitioner takeaway: Late-failure risk is usually a traceability problem before it is a technical problem, so the safest posture is to lock configuration early and keep the evidence synchronized with every material product change.

Related resources from NHI Mgmt Group

• What are the signs that a PowerShell 7 installation is likely to fail or become unreliable?
• What are the signs that an authentication policy is likely to fail after deployment?
• What are the signs that an OAuth login flow is misconfigured or likely to fail in production?
• What are the signs that a container base image change is likely to fail?