Start with the report file, the scanner property, and the scanner run order. If the report does not exist, uses the wrong format, or cannot be resolved from the scanner working directory, the problem is configuration. If the report is present and parsed, then investigate test scope, branch depth, and assertion quality.
Configuration vs code: the first split that saves time
Coverage problems usually fail in one of two places: the scanner never receives a valid report path, or the test suite genuinely did not produce enough signal to measure. The fastest triage is to verify the report file, the scanner property that points to it, and the order in which the scanner runs relative to test execution. That separates wiring faults from real test-quality issues.
If the scanner cannot resolve the report from its working directory, or the report is missing or malformed, you are looking at configuration rather than coverage logic. If the report is found and parsed, the question shifts from “did the tool run?” to “did the tests exercise enough code?”
That distinction matters because teams often spend time changing assertions, branch tests, or test depth when the scanner never had a usable input in the first place. Configuration defects can make healthy tests look broken, while a real coverage gap can hide behind a correctly wired pipeline.
What a parsed report tells you about the failure mode
Once the report is present and successfully parsed, the scanner has already crossed the infrastructure boundary. At that point, low coverage usually reflects test scope, branch depth, or weak assertions that do not drive execution through the code paths you expected. In other words, the measurement is now probably accurate, and the code under test is what needs attention.
A useful mental model is that configuration failures stop the data from arriving, while code-quality failures let the data arrive and then expose shallow tests. Missing files, wrong formats, or bad path resolution point to the measurement pipeline. Sparse branch coverage or repeated unexecuted assertions point to the test design.
Secure by Design guidance is relevant here because build-time and pipeline defaults should make miswiring obvious rather than silent. A coverage system that fails closed on missing input is easier to trust than one that quietly reports misleading numbers.
How to isolate the root cause in practice
Start with the simplest checks: does the report exist where the scanner expects it, does the file format match what the scanner reads, and is the scanner looking in the same workspace or container path that produced the report? If any of those fail, fix the pipeline before changing tests.
Next, confirm that the scanner actually runs after test execution and not before it. Run order problems often masquerade as low coverage because the scanner is reading an old report, an empty report, or no report at all. After that, examine whether the test suite exercises multiple branches and error paths, not just the happy path.
For teams standardising the measurement layer, NIST SP 800-53 Rev 5 Security and Privacy Controls is useful for thinking about configuration management, integrity, and auditability around build outputs. The control lens is simple: if the report path, format, and execution order are not controlled, the metric cannot be treated as trustworthy.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0 and NIST SP 800-53 Rev 5 set the technical controls, while ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | PR.PS-01 — Configuration Management | Coverage measurement depends on correct scanner and report configuration. |
| Recommendation — Harden scanner configuration so coverage reports are produced and consumed predictably. | ||
| NIST SP 800-53 Rev 5 | CM-2 — Baseline Configuration | The scanner, report path, and execution order are configuration dependencies. |
| AU-2 — Event Logging | Parsed reports and scanner runs create evidence for coverage diagnosis. | |
| Recommendation — Define and enforce a known-good scanning configuration baseline. Log scanner execution and report ingestion to distinguish pipeline faults from test gaps. | ||
| ISO/IEC 27001:2022 | A.8.9 — Configuration management | The issue hinges on whether tool and pipeline settings are managed correctly. |
| Recommendation — Control scanner and build settings so coverage measurement remains reliable. | ||
Practitioner Guidance
What to verify: Confirm the scanner property points to the exact generated report path and that the path is valid in the scanner’s runtime environment, not only on the developer machine.
Decision rule: If the report is absent, unreadable, or in the wrong format, treat the issue as pipeline configuration first; if the report is parsed successfully, move to branch depth, assertion quality, and test scope.
Common mistake: Teams often try to “improve coverage” by editing tests before verifying that the scanner is actually consuming the latest report from the intended workspace.
Practitioner takeaway: Coverage troubleshooting is fastest when you separate transport from substance, first prove that the scanner can reliably read the right report, then judge whether the tests themselves are deep enough.
Related resources from NHI Mgmt Group
- How do security and platform teams know whether a connection pool is failing because of app design rather than database capacity?
- How do security teams know whether their detection coverage is failing during holiday periods?
- How should security teams decide whether JIT access is safe for non-human identities?
- How do teams know whether infrastructure as code is increasing privilege risk?