Why do AppSec programmes stall when teams only report scan counts?

Why This Matters for Security Teams

Scan counts are a volume metric, not a risk metric. They can increase while exposure stays flat, especially when the programme is not measuring coverage, fix rate, or whether findings are tied to the applications that matter most. That is why executive reporting built around “how many scans ran” often creates the appearance of progress without proving protection.

Current guidance from the NIST Cybersecurity Framework 2.0 is to track outcomes, not just activity, and the same logic applies to AppSec. The most useful question is not whether testing happened, but whether it reached the right code, found the right issues, and drove remediation in time to matter. NHIMG research on Ultimate Guide to NHIs shows how often organisations miss similar control gaps in identity programmes, where visibility without lifecycle action leaves risk unresolved.

In practice, many security teams discover the reporting problem only after release pressure has already exposed the gap between scan activity and actual reduction in business risk.

How It Works in Practice

A useful AppSec programme separates “testing performed” from “security improved.” That means leadership dashboards should show coverage of critical applications, vulnerability severity by asset, remediation aging, re-open rates, and whether findings are being validated after fixes. Scan counts can still be reported, but only as a supporting metric.

The practical shift is to measure the pipeline end to end:

• coverage: which repositories, services, and release paths are actually being tested

• signal quality: whether scans are finding actionable issues or only generating noise

• fix velocity: how quickly critical findings move to verified closure

• repeatability: whether the same issue reappears in later builds or branches

• scalability: whether testing still works when code volume, teams, or release cadence increases

This is where the programme often needs policy and workflow changes, not a bigger count of scans. The NIST Cybersecurity Framework 2.0 supports this outcome-based view by encouraging organisations to track how controls reduce risk over time. NHIMG’s State of Secrets in AppSec notes that the average time to remediate a leaked secret is 27 days, which is a strong example of why speed to fix matters more than raw detection volume.

Once teams measure these flows, they can see where the bottleneck lives: missed coverage, noisy findings, slow approvals, or developers who cannot act on tickets fast enough. These controls tend to break down in fast-moving CI/CD environments because reporting systems still count scans even when the same critical services are never reached.

Common Variations and Edge Cases

Tighter reporting often increases operational overhead, requiring organisations to balance better assurance against the cost of collecting and normalising the right data. That tradeoff is real, especially for teams with many product groups, mixed tooling, or legacy pipelines where testing fidelity varies by application.

There is no universal standard for AppSec scorecards yet, but current guidance suggests avoiding metrics that can be gamed. Scan counts, ticket totals, and raw vulnerability numbers are easy to produce and easy to misread. A more reliable model is to segment reporting by application criticality, exposure, and remediation status, then compare trends over time rather than single snapshots.

Edge cases matter. For example, a low scan count may indicate poor coverage, but it may also reflect a mature release process with fewer changes and better preventive controls. Likewise, a high vulnerability count may show deeper inspection, not worse security. The question is whether the data supports decision-making. For teams governing secrets and service credentials, the same principle applies: The State of Secrets in AppSec and Ultimate Guide to NHIs both reinforce that lifecycle control and remediation speed matter more than activity metrics alone.

When the programme is used to justify staffing or budget, leadership should ask whether the numbers prove reduced exposure, or only more scanning effort. In environments with outsourced development or heavily federated engineering, that distinction becomes especially difficult because scan ownership and fix ownership are often split across different teams.

Related resources from NHI Mgmt Group

• How should security teams measure AppSec success beyond scan counts?
• Why do small security teams often succeed with Zero Trust when larger programmes stall?
• What breaks when AppSec teams rely on scan severity alone?
• How should security teams prioritise AppSec findings when every scan produces thousands of alerts?