What breaks when build security only scans code and manifests before the pipeline runs?

Why This Matters for Security Teams

Scanning code and manifests before the pipeline runs only answers what was declared, not what actually executed. Build-time checks can validate package names, lockfiles, and policy intent, but they do not see post-install scripts, fetches from attacker infrastructure, or runtime access to mounted credentials. That gap matters because supply chain abuse often hides in the gap between declared dependencies and observed behaviour.

NHI Mgmt Group research shows why this is not a theoretical problem: 96% of organisations store secrets outside of secrets managers in vulnerable locations, and 79% have experienced secrets leaks, with tangible damage in most incidents. The Guide to the Secret Sprawl Challenge explains how easily credentials spread across pipelines and tooling, while the NIST Cybersecurity Framework 2.0 reinforces that detection and monitoring must extend beyond pre-deployment review.

In practice, many security teams discover the abuse only after a build has already exfiltrated secrets or pulled malicious code, rather than through intentional prevention.

How It Works in Practice

Pre-build scanning is useful, but it is not sufficient control coverage for modern pipelines. The attacker’s most effective move is often to behave normally until execution time. A package may pass manifest review, then run a script during install, reach out to external infrastructure, or inspect environment variables and credential files once the job starts. This is why runtime observation matters: it reveals the difference between a harmless declaration and a harmful action.

Effective pipeline defence combines static checks with execution-time controls. Teams should watch for network egress to unknown destinations, unexpected child processes, credential file access, and package manager hooks that trigger during install or post-install. Telemetry from build runners, container runtimes, and CI/CD systems helps correlate declared intent with actual behaviour. The CI/CD pipeline exploitation case study shows how compromised build workflows can become a launch point for downstream compromise. The OWASP Software Supply Chain Security Top 10 is a useful reference for prioritising these controls.

• Inspect runtime script execution, not just dependency manifests.
• Use short-lived secrets so a stolen token expires before it can be reused.
• Alert on outbound connections, especially from build containers.
• Restrict runner permissions so one compromised job cannot reach unrelated assets.
• Record provenance and execution telemetry together so reviewers can compare expected and actual behaviour.

This guidance tends to break down in highly dynamic build farms with shared runners and weak workload isolation because runtime signals become noisy and attribution is difficult.

Common Variations and Edge Cases

Tighter runtime control often increases pipeline overhead, requiring organisations to balance build speed against stronger observation and containment. That tradeoff is most visible in high-throughput CI environments, ephemeral container builds, and language ecosystems that rely heavily on install-time scripts. In those settings, there is no universal standard for how much runtime telemetry is enough, so current guidance suggests layering controls rather than relying on one gate.

One common edge case is trusted internal packages that still execute unsafe behaviour because they inherit broad runner permissions or weak secret hygiene. Another is monorepo builds, where a single pipeline may touch many services and make least-privilege hard to enforce cleanly. The Reviewdog GitHub Action supply chain attack is a reminder that even popular automation can become a secret-exposure path when execution is not observed. Current best practice is to treat build-time scanning as one layer, then pair it with runtime policy, egress control, and aggressive secret rotation. This is also where the NIST Cybersecurity Framework 2.0 supports continuous monitoring rather than one-time approval.

For teams operating at scale, the real question is not whether a manifest is clean, but whether the pipeline behaved safely after the manifest was approved.

Related resources from NHI Mgmt Group

• What breaks when security testing only runs after code is committed in AI-assisted workflows?
• What breaks when security testing still depends on periodic scans in an AI-driven delivery pipeline?
• What breaks when security policy is enforced only after AI-generated code reaches the pipeline?
• What should security teams check before using chat to build provisioning workflows?