Configuration-only monitoring misses the actual execution path, which is where secrets are read, commands are launched, and network traffic is generated. A workflow can look clean on paper and still be abused at runtime by a compromised dependency, tampered action, or malicious step. Security teams need behavioural telemetry to catch what the YAML file cannot show.
What breaks when you only watch the pipeline config?
Configuration-only monitoring gives you a static view of intent, not a live view of execution. In CI/CD, that means you can miss the very moments when secrets are consumed, trusted steps are invoked, or runtime behaviour diverges from the YAML. The practical failure is false confidence: the pipeline can look compliant while the actual job is already being abused.
That gap matters because many compromises happen after the config is parsed. A pinned workflow can still call an altered action, a dependency can change behaviour, and a benign-looking step can become the point where credentials are exposed or commands are redirected.
Why runtime visibility matters more than clean configuration
CI/CD systems are dynamic execution environments. The file in the repository describes intended control flow, but the actual attack surface includes the runner, action code, environment variables, network egress, artifact handling, and any dependency fetched during the job. Monitoring only the declarative layer misses the evidence that shows whether the intended controls were actually enforced.
This is why behavioural telemetry is the decisive layer for this subject. It lets teams see which process touched which secret, which network destination a step reached, and whether a trusted workflow began doing something the config never advertised. For build provenance and supply-chain assurance, SLSA is the clearest external reference for why build integrity has to be verified from execution evidence, not just repository state.
The distinction also explains why a “safe” config can still produce an unsafe build. Runtime compromise often appears as an ordinary pipeline step until a behavioural signal reveals secret reads, unexpected shell execution, or an external callback that should never have happened in that job context.
What practitioners should watch instead of the YAML alone
The right monitoring model combines configuration review with execution-time signals. In practice, that means watching for secret access, process creation, unexpected outbound traffic, altered step lineage, runner persistence, and token use that does not match the declared workflow intent. Those are the clues that the pipeline’s real authority path has changed.
Internal case studies make the point sharply. The ArtiPACKED 2024 research showed how runtime artifacts and tokens can leak from GitHub Actions even when the workflow definition itself looks ordinary. Likewise, the tj-actions/changed-files compromise 2025 demonstrates that a compromised action can turn a normal pipeline into a secret-exfiltration event without any obvious static config change.
A useful rule is simple: if a control only tells you what was supposed to happen, it is incomplete for CI/CD security. You also need to know what the runner actually did, because that is where compromise, exfiltration, and abuse become visible.
Risk and Threat Considerations
Configuration-only monitoring creates blind spots at the exact point where CI/CD is most sensitive, the runtime. Attackers exploit that gap by tampering with actions, abusing dependencies, or using a legitimate workflow step to read secrets and launch follow-on activity that the YAML never reveals.
Failure mechanism: The pipeline definition remains unchanged while the execution path is subverted through compromised third-party code, injected commands, stolen tokens, or unexpected network and file activity on the runner.
Impact: Teams may miss secret theft, unauthorized publishing, poisoned builds, or lateral movement into adjacent systems, and they may continue trusting a pipeline that is already executing attacker-controlled behaviour.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP API Security Top 10 addresses the attack and risk surface, while SLSA and NIST SP 800-53 Rev 5 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| SLSA | Supply-chain Levels for Software Artifacts | Build provenance depends on execution evidence, not config alone. |
| Recommendation — Verify build provenance and integrity from runtime evidence, not repository state alone. | ||
| NIST SP 800-53 Rev 5 | AU-6 — Audit Record Review, Analysis, and Reporting | Runtime telemetry is needed to detect pipeline abuse beyond static config. |
| SI-4 — System Monitoring | Behavioural monitoring is required to catch malicious runtime changes in pipelines. | |
| Recommendation — Review CI/CD audit records for secret access, execution anomalies, and unexpected network activity. Monitor runner activity, step execution, and egress behaviour for suspicious CI/CD execution. | ||
| OWASP API Security Top 10 | API8 — Security Misconfiguration | Static configuration can look correct while runtime behaviour remains exploitable. |
| Recommendation — Validate that operational enforcement matches declared security configuration. | ||
Practitioner Guidance
What to prioritise: Instrument the runner and job execution layer first, because that is where secret access, command execution, and egress misuse become observable. Keep config review, but treat it as the starting point, not the control boundary.
What to verify: Confirm that you can correlate workflow steps to process activity, secret access, and outbound connections for each build. If you cannot reconstruct those runtime events, your monitoring is not yet strong enough to detect pipeline abuse.
Common mistake: Treating pinned versions, code review, or clean YAML as proof that the pipeline is safe. Those controls reduce risk, but they do not prove that the executed job stayed inside its expected trust boundary.
Practitioner takeaway: In CI/CD, security failures usually show up in execution, not in the configuration file, so the control objective is to make runtime behaviour visible enough to prove when the pipeline has stopped doing what the YAML says.