Join our Newsletter — 33% off our NHI Course

Notifications
Clear all

Sha1-Hulud in CI/CD pipelines: what runtime monitoring caught


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 18004
Topic starter  

TL;DR: Anomalous runtime connections to bun.sh and oss.trufflehog.org helped detect the Sha1-Hulud npm supply chain attack in Backstage, according to StepSecurity. The malware also attempted runner persistence through GitHub API registration, reinforcing that pre-deployment scanning and lock-file checks are not enough when trusted packages are hijacked at runtime.

NHIMG editorial — based on content published by StepSecurity: How Harden Runner Detected the Sha1-Hulud Supply Chain Attack in CNCF's Backstage Repository

Questions worth separating out

Q: What breaks when dependency confusion is allowed in CI/CD pipelines?

A: Build systems can install malicious public packages instead of trusted internal dependencies, which turns normal package installation into an execution path for secret theft.

Q: Why do build servers and CI runners increase supply chain risk?

A: Build servers and CI runners often contain reusable secrets, access to artifact repositories, and permissions that ordinary endpoints do not need.

Q: How do security teams know if a workflow is behaving outside its intended boundary?

A: They compare observed behaviour against a known baseline for the workflow, including destinations, processes, and registration activity.

Practitioner guidance

  • Instrument workflow runtime with baseline anomaly detection Establish expected outbound destinations for each GitHub Actions workflow and alert on any new network target, especially during dependency install and test steps.
  • Treat CI/CD runners as privileged non-human identities Inventory runner tokens, repository permissions, and attached secrets as part of NHI governance, then reduce standing access to the minimum needed for each pipeline stage.
  • Block unexpected external downloads during package execution Apply egress policy enforcement so workflows cannot fetch runtimes, scanners, or other executables from unapproved domains such as bun.sh or similar endpoints that never belong in routine builds.

What's in the full article

StepSecurity's full blog covers the operational detail this post intentionally leaves for the source:

  • Interactive detection traces for the Backstage workflow runs that show exactly which network destinations were flagged as anomalous.
  • Enterprise-tier visibility into HTTPS requests, process trees, and command-line arguments, which helps teams move from alerting to root-cause analysis.
  • Evidence of how the malware attempted self-hosted runner registration through the GitHub API, showing the persistence path in more detail.
  • Step-by-step recommendations for open source and enterprise teams that need to operationalise runtime monitoring across CI/CD.

👉 Read StepSecurity's analysis of the Sha1-Hulud supply chain attack in Backstage →

Sha1-Hulud in CI/CD pipelines: what runtime monitoring caught?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 3 months ago
Posts: 17593
 

Runtime behaviour is now the decisive control plane for CI/CD supply chain defence. Static scanning still matters, but it does not stop a trusted package from executing malicious code after installation. The Backstage case shows why baseline anomaly detection is stronger than assumption-based trust in build-time artefacts. Teams should treat workflow telemetry as a primary control, not an optional supplement.

A question worth separating out:

Q: What should teams do when a CI/CD workflow attempts persistence through runner registration?

A: Contain the workflow, revoke the affected runner credentials, and review whether the repository can register additional runners or access shared secrets. Then verify whether the registration API call was isolated or part of a broader compromise chain that touched other jobs or repositories.

👉 Read our full editorial: Runtime monitoring exposed the Sha1-Hulud npm supply chain attack



   
ReplyQuote
Share: