What breaks when application security is based only on code and manifest analysis?

The main failure is that teams lose sight of whether a weakness is actually reachable at runtime. A dependency can look risky on paper while never being loaded, executed, or accessed in the live application. That gap pushes teams toward expensive cleanup work that does not reduce real attack surface and can undermine confidence in the program.

What the code-only view misses at runtime

Code and manifest analysis can tell you what is declared, packaged, or theoretically reachable, but it cannot prove what the application actually loads, invokes, or exposes in production. That matters because modern applications often contain optional modules, environment-specific features, conditional imports, feature flags, and dead dependencies that never become part of the real attack surface.

Once teams confuse declared inventory with runtime behaviour, they start treating every risky-looking dependency as equally actionable. The result is noisy findings, wasted remediation effort, and blind spots around the controls that actually govern execution, such as runtime configuration, exposed routes, loaded libraries, and live service-to-service access paths.

That gap is visible in the broader secrets problem too: NHI Mgmt Group reports that 30.9% of organisations store long-term credentials directly in code, which shows why static inspection is useful, but also why it is incomplete when used alone. Code can reveal embedded secrets, yet it still does not answer whether a given secret, package, or path is active in the running system. Guide to the Secret Sprawl Challenge

Static-only analysis also struggles with transitive and conditional risk. A package may appear dangerous because of a CVE, but if the vulnerable component is never loaded in the deployed build or is unreachable from any exposed execution path, the practical exposure is far lower than the report suggests. Conversely, a small-looking manifest can hide a real issue if runtime configuration activates a feature, plugin, or endpoint that the manifest alone does not make obvious.

Why false positives and false negatives both increase

When analysis stops at code and manifests, false positives rise because scanners flag every declared dependency or embedded secret without knowing whether the weakness can be exercised. That encourages broad cleanup campaigns that consume engineering time but do not reduce meaningful attack surface.

False negatives rise for the same reason, because runtime state often determines whether a weakness is reachable. The application may only expose the vulnerable path in a specific environment, under a particular configuration, or after a deployment change that never appears in the source tree. This is why practitioners need runtime validation in addition to software composition and manifest review, not as a replacement for them.

Application security standards reflect this separation of concerns. OWASP ASVS explicitly covers authentication, session handling, access control, and validation, all of which depend on how the application behaves when it is running, not only on what appears in source control. For broader testing guidance, OWASP Web Security Testing Guide helps teams test the live attack surface rather than assuming the manifest tells the whole story.

Manifest review is still valuable for build-time hygiene, supply-chain exposure, and obvious mispackaging, but it should be treated as one input to an exposure decision, not the decision itself. If your program cannot distinguish “declared” from “reachable,” it will overprioritise some issues and miss others that matter more.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

CIS Controls v8 provides the primary governance reference for this topic.

Framework	Control / Reference	Relevance
CIS Controls v8	CIS 2 — Inventory and Control of Software Assets	Declared software inventory must be reconciled with what is actually present and used.
	CIS 16 — Application Software Security	Application security needs testing of deployed behaviour, not just code and manifests.
Recommendation — Compare software inventory to runtime use before prioritising remediation. Assess running application behaviour to identify reachable weaknesses.

Practitioner Guidance

What to prioritise: Anchor the decision on runtime reachability. If a weakness is only present in the dependency graph or a manifest entry, validate whether the component is loaded, invoked, or exposed in the deployed environment before assigning remediation priority.

What to verify: Correlate static findings with build artefacts, deployment configuration, process inventory, exposed endpoints, and authenticated execution paths. A finding should be considered materially actionable only when you can show a path from declaration to runtime use.

Common mistake: Treating “present in the repo” as equivalent to “exploitable in production.” That shortcut drives noisy backlogs and weakens trust in the security programme when teams repeatedly burn effort on items that never mattered operationally.

Practitioner takeaway: The right question is not whether the codebase contains something risky, but whether the running application can actually be made to use it in a way that changes attack surface.

Risk and Threat Considerations

Static-only analysis creates both governance risk and exposure risk because it can misstate the real attack surface. Defenders may spend time removing dormant components while missing the runtime paths that attackers can actually reach.

Failure mechanism: The failure is an evidence gap between declared software content and live execution state, which allows unreachable issues to look urgent and reachable issues to stay hidden until deployment or exploitation.

Impact: This can inflate remediation cost, distort risk prioritisation, and leave organisations with a false sense of coverage over the parts of the application that are truly exposed.