Late failures usually show up when the Security Target does not match the shipped product, the referenced algorithms lack active validation, or the evidence package cannot prove configuration control. Another warning sign is weak assurance continuity, where patching and library changes are not tracked against the certified baseline. Those gaps tend to surface when assessors start probing the cryptographic boundary.
What late-failure signs mean in a Common Criteria evaluation
Late failure usually means the evaluation has moved past paper quality and into evidence scrutiny, where mismatches become expensive to repair. The most reliable early warning is a gap between what the Security Target claims and what the delivered product actually contains, especially when configuration, crypto, or platform scope has shifted since the draft was written. That gap tends to become visible only once the evaluator begins tracing evidence end to end.
A second signal is that the assurance story is not closed under change. If patching, dependency updates, build options, or library substitutions are happening faster than the certification baseline is being maintained, assessors will eventually find that the evidence no longer describes the product under review. At that point, the issue is not a missing document, but a broken chain of trust between design, build, and delivered configuration.
The third sign is weak proof around the cryptographic boundary and related supporting assumptions. When the boundary is unclear, when validated algorithms are referenced without current validation status, or when the evaluation relies on “intended” configuration rather than reproducible configuration, the review often runs smoothly until the assessor asks for exact artefacts. That is where late-stage rework appears.
Where late-failure risk accumulates in the evidence package
Late failure is rarely caused by one dramatic defect. It is usually the accumulation of small inconsistencies: a Security Target that describes one product revision, test evidence that reflects another, or design documentation that omits a dependency now inside scope. Those inconsistencies are survivable early, but they become blocking when the assessor needs one coherent assurance chain.
Configuration control is often the decisive pressure point. If the product can be rebuilt in more than one way, or if the certification baseline is not the same artifact the team is shipping, the evaluator cannot safely rely on the evidence set. Likewise, if third-party libraries, patches, or cryptographic components changed after the evidence was frozen, the evaluation can fail not because the product is weak, but because the assurance case is no longer provable.
Algorithm validation is another common trigger. When the evaluation depends on cryptographic claims, the team needs to show that the algorithms, modules, and operating modes are still the ones covered by the validation story. If that linkage is stale, missing, or ambiguous, the failure tends to surface late because the issue only becomes obvious when the evaluator checks the exact boundary and configuration state.
What practitioners should watch before the assessor does
Three practical checks usually separate a recoverable review from a late collapse. First, verify that the shipped build, the Security Target, and the evidence package all describe the same thing. Second, verify that every change since the baseline is either absorbed into the certification story or explicitly excluded. Third, verify that cryptographic components, dependencies, and platform assumptions are still aligned with the claimed assurance scope.
It also helps to treat the evaluation package as a living control surface rather than a final binder. If engineering teams are still changing libraries, toggling build flags, or adjusting deployment topology while the evidence is being prepared, the certification team needs a formal point of control. Without that, the assessor discovers drift first, and late discovery almost always means schedule impact.
What to verify: Confirm that the baseline is reproducible from source, build, and deployment artefacts, and that the exact configuration under test is the one described in the Security Target. If that cannot be proven quickly, assume the evaluation is already at risk.
Decision rule: If a patch, library upgrade, or crypto change alters the certified boundary, treat it as a certification-impacting change until the evidence package is updated and rechecked. Do not wait for the assessor to raise the mismatch.
Common mistake: Teams often assume that strong functional test results can compensate for weak configuration proof. In common criteria, they cannot. The evaluation can still fail if the assurance chain does not show control, continuity, and exact product identity.
Practitioner takeaway: Late-failure risk is usually a traceability problem before it is a technical problem, so the safest posture is to lock configuration early and keep the evidence synchronized with every material product change.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST SP 800-53 Rev 5 sets the technical controls, while ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | CM-2 — Baseline Configuration | Common Criteria late failure often reflects baseline drift and mismatched product evidence. |
| CM-3 — Configuration Change Control | Late-stage evaluation failure is frequently triggered by unmanaged patches, library, or build changes. | |
| SA-11 — Developer Testing and Evaluation | The evaluation depends on test and evidence artefacts proving the claimed product behavior and boundary. | |
| Recommendation — Establish and maintain a certified baseline for the evaluated product and its evidence set. Require formal review before any change can alter the evaluated configuration. Retain test evidence that matches the exact evaluated build and security boundary. | ||
| ISO/IEC 27001:2022 | A.8.9 — Configuration management | Configuration control is central to keeping the evaluated product aligned with the certified baseline. |
| A.8.32 — Change management | Untracked changes to code, dependencies, or deployment can break the assurance case late in review. | |
| Recommendation — Control configurations so the shipped product matches the documented assurance scope. Assess and approve changes for their impact on certification evidence before release. | ||
Related resources from NHI Mgmt Group
- What are the signs that a PowerShell 7 installation is likely to fail or become unreliable?
- What are the signs that an authentication policy is likely to fail after deployment?
- What are the signs that an OAuth login flow is misconfigured or likely to fail in production?
- What are the signs that a container base image change is likely to fail?
