Most release processes check the code thoroughly and then let almost anyone, or any token, push the result into production. The release stage is where a tested build becomes a running service, and it is the point at which segregation of duties, approval evidence and artefact integrity either hold or quietly fall away. This guide covers the release stage only: environment promotion, approval and deployment gates, who may approve what, emergency and hotfix releases, rollback, the risk carried by feature flags, signed artefacts and the audit evidence that change auditors ask for. It is written for platform engineers, release managers, security engineers and the IT audit teams who test change controls. For the whole pipeline at a higher level, see the DevSecOps Guide.
Key takeaways
- Gate the environment, not the pipeline file. Approval rules attached to a protected environment (GitHub environments, GitLab protected environments, Azure Pipelines environment checks) survive edits to the workflow YAML; rules written inside the YAML do not.
- Self-approval is the default gap. GitHub only blocks the deployer from approving if "prevent self-reviews" is switched on, and Azure Pipelines lets you permit or restrict approvers approving their own runs. Check the setting on every production environment.
- Promote the same artefact, by digest. Rebuilding for each environment means production runs something nobody tested. Build once, sign or attest once, and deploy the identical digest through every stage.
- Emergency releases need a faster path, not no path. Pre-approve the break-glass route, log every use, and require a retrospective approval within a fixed window. Untracked admin bypass is what change auditors find first.
- Release rights are an identity problem. Deploy tokens, cloud roles used by pipelines and maintainer accounts that can edit releases are the real control boundary. Scope them per environment and make releases immutable.
What the release stage includes and why it needs its own gates
Build and test gates ask whether code is acceptable. Release gates ask whether this is the exact artefact that passed testing, going to the right place at an acceptable time, agreed by someone other than the author, with proof. Merging the two is the root of most release control failures.
| Release activity | Security question it answers | Typical control |
|---|---|---|
| Environment promotion (dev to test to staging to production) | Is the artefact in production the one that was tested? | Immutable artefact repository, promotion by digest, environment-scoped credentials |
| Approval gate | Has an independent person accepted the change and its risk? | Required reviewers on a protected environment, self-approval blocked |
| Deployment gate (automated checks) | Are the technical preconditions met right now? | Branch restriction, change window, open-incident check, policy evaluation of the image |
| Artefact integrity | Was this built by our pipeline from our source, and unchanged since? | Signature or provenance attestation verified at deploy time |
| Emergency and hotfix release | Can we move fast without losing accountability? | Pre-defined break-glass route, logged bypass, retrospective approval |
| Rollback | Can we return to a known good state without introducing new risk? | Retained signed prior releases, tested rollback, rollback uses the same gates |
| Feature flags | Is code being activated in production without a change record? | Flag changes treated as production changes, flag debt removed |
| Audit evidence | Can an auditor reconstruct who changed what, when and with whose approval? | Platform audit logs, deployment history, linked change tickets, retention |
NIST SP 800-53 Rev. 5 places most of this under CM-3 Configuration Change Control, CM-4 Impact Analyses and CM-5 Access Restrictions for Change, with AC-5 Separation of Duties for the approver rule and CM-14 Signed Components for artefact integrity.
Approval and deployment gates on the main platforms
Platforms are listed alphabetically. The three most common CI/CD platforms all attach gates to a deployment target rather than to the pipeline definition, which is the property you want: a developer who can edit the workflow file cannot remove the gate.
| Capability | Azure Pipelines (environments and resources) | GitHub Actions (environments) | GitLab (protected environments) |
|---|---|---|---|
| Manual approval | Approvals check: users or groups; for a group, one member's approval suffices; optional instructions for approvers | Required reviewers: up to six users or teams; one approval lets the job proceed | Approval rules with a required number of approvals; deployments are blocked until all required approvals are given |
| Blocking self-approval | Option to permit or restrict approvers approving their own runs | "Prevent self-reviews": the user who initiated the deployment cannot approve, even if listed as a reviewer | Triggerer cannot approve by default; "Allow pipeline triggerer to approve deployment" turns this off |
| Source restriction | Branch control check (fully qualified refs such as refs/heads/main, can require branch protection) | Deployment branches and tags: no restriction, protected branches only, or selected branches and tags | Protected environment defines who may deploy; approval rules define who may approve |
| Timing | Business hours check; approval timeout marks the stage skipped; deferred approval | Wait timer of 1 to 43,200 minutes (30 days) | Approval does not start the job; someone must run it afterwards |
| Automated policy gates | Evaluate artifact, Invoke Azure Function, Invoke REST API, Query Azure Monitor Alerts, Required template, Exclusive lock | Custom deployment protection rules powered by GitHub Apps | Blocked deployments are shown as "blocked" in the environment view and the API status field |
| Administrator override | Approver list is fixed when checks start, so later changes are not picked up mid-run | Administrators can bypass by default; environments can be set to disallow bypassing for all deployments | A GitLab administrator can approve or reject any deployment |
| Secrets tied to the gate | Checks apply to protected resources such as service connections and variable groups | Environment secrets are unavailable to a job until a required reviewer approves | Environment-scoped CI/CD variables (configure per protected environment) |
Availability: Azure Pipelines: the ServiceNow Change Management check needs a separate Marketplace extension. GitHub: on Free, Pro and Team plans, required reviewers, wait timers, custom protection rules and the admin bypass setting apply to public repositories only, per GitHub's documentation. GitLab: deployment approvals require Premium or Ultimate. Confirm your edition before designing a control around a feature.
- Put production credentials behind the gate: on GitHub, an environment secret or an OIDC trust condition scoped to the production environment means an unapproved job never receives the credential. On Azure Pipelines, put the approval on the service connection, not only on the stage. See the GitHub Actions OIDC Federation Hardening Guide.
- Gate on system health: a check that fails when monitoring alerts are firing (Query Azure Monitor Alerts, or a custom protection rule calling your incident tool) stops a release landing on top of an open incident.
- Size approval counts realistically: GitLab warns that the required count must be below the number of users allowed to deploy, or deployments stall.
Segregation of duties for deployments
The classic rule is that the person who writes a change should not be the person who alone approves and deploys it to production. Automation changes how the rule is met, not whether it applies. See the Segregation of Duties Guide for the general model.
| Role | May | Should not |
|---|---|---|
| Author (developer) | Commit, open a merge request, trigger non-production deploys | Approve their own production deployment; hold standing production deploy credentials |
| Code reviewer | Approve the merge into the release branch | Be the same person as the author (enforce through branch protection) |
| Release approver | Approve promotion to production for a defined service | Edit environment protection rules for the environments they approve |
| Platform administrator | Configure environments, rules and runners | Routinely deploy application changes; use bypass outside the emergency route |
| Pipeline identity (service principal, deploy token, OIDC role) | Deploy approved artefacts to one environment | Be shared across environments; be usable from an unprotected branch |
- Small teams: where only three engineers exist, full separation is unrealistic. A defensible compromise is peer approval by any other engineer, automated gates that cannot be skipped, and a weekly review of the deployment log by someone outside the team. Document it as a compensating control.
- Large organisations: map approvers to services through groups owned by the service, not by the platform team, and review group membership quarterly as part of access certification. See the Access Reviews Guide.
- Automated approval is still approval: a custom protection rule that approves when tests, scans and a change ticket all pass can satisfy the intent of separation, because the author cannot alter the rule. Record that design in the control description so auditors test the rule rather than looking for a human click.
Signed artefacts and promotion by digest
The release gate is only meaningful if the thing approved is the thing deployed. Three practices close that gap.
- Build once: produce the artefact (container image, package, binary) a single time and store it in a repository that refuses overwrites of a published version.
- Sign or attest at build time: GitHub artifact attestations use Sigstore and, on their own, provide SLSA v1.0 Build Level 2; building in a shared reusable workflow can reach Build Level 3, per GitHub's documentation. GitHub's guidance is to sign software you release (binaries, packages, manifests) and not to sign frequent test builds or individual source files. See the Sigstore Guide and the SLSA and Build Provenance Guide.
- Verify at deploy time, reference by digest: the deploy step, or an admission controller, verifies the signature or provenance and deploys by immutable digest (sha256) rather than a mutable tag such as latest. GitHub is explicit that generating attestations brings no benefit unless they are verified, and that an attestation is not a guarantee the artefact is safe: it links the artefact to source and build instructions, and you decide the policy.
Release files that can be edited after publication undo all of this. In the OVHcloud CLI Compromise 2026, binaries for already published versions were replaced through a contributor's account; OVHcloud said release immutability was disabled, so the swap bypassed CI/CD entirely, and a user's pipeline caught it only because it pinned a SHA-256 hash. Turn on release or tag immutability where the platform offers it, and pin hashes when you consume releases. For deeper coverage see the OWASP CI/CD Top 10 CICD-SEC-9 Improper Artifact Integrity Validation Guide and the Code Signing Security Guide.
Emergency and hotfix releases
Every organisation has a way to ship during an outage. The question is whether it is designed or improvised. An improvised path usually means an administrator bypasses environment rules, and nothing links that bypass to an incident.
- Define the route in advance: a separate "emergency" approval group with a small, named membership and a lower approval count (for example one approver who is not the author), still behind the same environment, the same artefact verification and the same credential scope.
- Keep integrity checks on: speed comes from fewer humans, not from skipping signature verification or the branch restriction. A hotfix is built by the same pipeline from a hotfix branch that matches the deployment branch rule.
- Restrict bypass: on GitHub, disallow administrator bypass on production environments and use the emergency group instead; where bypass must exist, alert on it.
- Retrospective approval: require the normal approvers to review every emergency release within a fixed window (one or two working days is common) and link it to the incident record. Track the share of releases that went through the emergency route: a rising share means the normal route is too slow.
Rollback and feature flags
Rollback and flags are both ways to change production behaviour quickly, and both are regularly left outside change control.
| Mechanism | Security risk | Control |
|---|---|---|
| Redeploy previous release | Reintroduces vulnerabilities fixed in the newer release; prior artefact may no longer be retained or verifiable | Retain signed prior releases for the rollback window; run the rollback through the same environment gate; check known vulnerabilities before rolling back past a security fix |
| Feature flag enabled in production | Activates code paths without a deployment, approval or change record; flags may also expose unfinished features | Treat production flag changes as changes: role-based access to the flag console, approval for flags on security-relevant paths, audit log export |
| Long-lived or repurposed flag | Dead code behind an old flag is reactivated by reuse of the flag name | Expiry date per flag, removal of flag and code after rollout, never reuse a flag name |
A staged rollout (canary rings, percentage rollout, progressive regions) limits the damage of a bad release whether the cause is a bug or a malicious change, which is why the Third-Party Concentration Risk Guide uses the July 2024 CrowdStrike outage as its case study.
Where identity fits
Every release gate eventually reduces to "which identity is allowed to put bits into production". Those identities are mostly non-human: service principals behind Azure service connections, OIDC roles assumed by GitHub workflows, GitLab deploy tokens, registry push credentials, and the personal access tokens of maintainers who can edit releases.
- One identity per environment: the staging deploy identity must not be able to write to production. Scope OIDC trust to the environment claim, not just the repository.
- No standing human deploy rights: humans approve; the pipeline deploys. Human production access for incidents goes through privileged access with time limits.
- Protect publishing identities: accounts that can create or edit releases, push tags or publish packages need phishing-resistant MFA and short-lived tokens. See the CI/CD Pipeline Identity Security Guide.
- Detect the abuse case: MITRE ATT&CK tracks adversary use of deployment systems as T1072 Software Deployment Tools and tampering with distributed software as T1195.002 Compromise Software Supply Chain.
Audit evidence for change controls
Auditors testing IT general controls over change management, including for SOX-regulated companies, typically sample production changes and ask for evidence that each was authorised, tested and approved by someone other than the developer, and that only authorised people could deploy. Pipelines can produce that evidence automatically if you design for it.
| Auditor question | Evidence source |
|---|---|
| Was the change authorised? | Change ticket ID in the commit, merge request or deployment record; deployment approval with approver identity and timestamp |
| Was it tested? | Pipeline run with test and scan results for the same artefact digest that was deployed |
| Was approval independent of the author? | Self-review prevention setting (configuration export) plus approver versus triggerer comparison for the sample |
| Who can deploy to production? | Environment protection configuration, deploy identity permissions, approver group membership with review evidence |
| Were emergency changes controlled? | List of bypasses and emergency-route deployments, each with incident link and retrospective approval |
| Was the deployed artefact the tested one? | Signature or attestation verification log at deploy time, digest recorded in the deployment |
- Export and retain: platform audit logs have finite retention. Stream deployment and environment-configuration events to your log platform and keep them for the period your audit or regulator requires.
Implementing release gates: a phased rollout
- Inventory (weeks 1 to 2): list every production deployment path, including manual ones, scripts on laptops and console changes. Record which identity each uses.
- Protect production environments (weeks 2 to 4): define a protected environment per production target, restrict deploying branches, move production credentials into environment scope, and enable self-approval blocking. Start with a pilot of two or three services.
- Observe before enforcing: for automated policy gates (artefact policy, signature verification), run in audit or warn mode first and review what would have failed. Fix false positives before switching to block.
- Introduce the emergency route (week 4): create the emergency approver group, disable general administrator bypass on production, and rehearse a hotfix in a game day.
- Add integrity (weeks 4 to 8): sign or attest release artefacts, verify at deploy, deploy by digest, turn on release immutability, and retain prior releases for rollback.
- Bring flags into scope: restrict production flag changes, export flag audit logs, set flag expiry.
- Automate evidence and expand: link change tickets, stream logs, then roll the pattern out service by service.
Rollback of the controls themselves: each gate is configuration on the environment, so if a new check blocks legitimate releases you can switch it back to warn mode or remove it for that environment while you fix the cause.
Detection and measurement
- Monitor: changes to environment protection rules, approver lists and branch rules; administrator bypass events; deployments by identities other than the expected pipeline identity; release or tag edits after publication; production flag changes on security-relevant flags; control-plane changes that do not correlate with a pipeline deployment.
- Self-approval rate: share of production deployments where triggerer and approver are the same identity. Target zero outside documented small-team exceptions.
- Emergency route share: emergency releases as a percentage of all production releases, and the percentage with retrospective approval inside the window (target 100%).
- Verified deploy rate: percentage of production deployments where signature or provenance was verified and the digest recorded.
- Out-of-band change count: production changes seen in control-plane logs with no matching deployment record.
Framework mapping
| Release control | NIST SP 800-53 Rev. 5 | NIST CSF 2.0 | ISO/IEC 27001:2022 Annex A |
|---|---|---|---|
| Approval and change records | CM-3 Configuration Change Control, CM-4 Impact Analyses | PR.PS-01 | A.8.32 Change management |
| Who may deploy | CM-5 Access Restrictions for Change | PR.PS-01 | A.8.32, A.8.2 Privileged access rights |
| Approver independence | AC-5 Separation of Duties | PR.PS-06 | A.5.3 Segregation of duties |
| Environment promotion | SA-10 Developer Configuration Management | PR.PS-06 | A.8.31 Separation of development, test and production environments |
| Signed artefacts, integrity | CM-14 Signed Components, SI-7 Software, Firmware, and Information Integrity | PR.PS-06 | A.8.25 Secure development life cycle |
NIST's Secure Software Development Framework (SP 800-218) covers the same ground in its "Protect the Software" practices, including providing a way to verify release integrity and archiving each release. For the policy wording, use the Change Management Policy Template.
Common failure patterns
- Gate in YAML only: the approval step lives in the workflow file, so any developer with write access can delete it in the same merge request that ships the change.
- Self-approval left on: required reviewers are configured but the deployer is one of them and "prevent self-reviews" is off, so separation exists only on paper.
- Production credentials outside the gate: the production cloud key sits in a repository or organisation secret, so an ungated job or a branch that is not protected can still deploy.
- Mutable releases and tags: published releases can be edited, so a compromised maintainer account swaps binaries after every gate has passed, as in the OVHcloud CLI case above.
- Rebuild per environment: each stage rebuilds from source, so the production binary was never tested and dependency resolution may differ.
- Feature flags as a side door: large behaviour changes are enabled from a flag console that dozens of people can use, with no approval and logs kept for 30 days.
Practitioner checklist
- Every production target is a protected environment with approval rules that the workflow file cannot override.
- Self-approval is blocked on every production environment, and the setting is exported as audit evidence.
- Production credentials are scoped to the production environment (environment secrets, OIDC trust on the environment claim or approvals on the service connection).
- Deploying branches and tags are restricted to protected release branches.
- Release artefacts are built once, signed or attested, verified at deploy and deployed by digest; release immutability is on.
- An emergency route exists with a named group, logged use and retrospective approval inside a fixed window; general administrator bypass is disabled on production.
- Prior signed releases are retained for rollback, and rollback runs through the same gates.
- Production feature flag changes require role-based access, approval for security-relevant flags and exported audit logs; flags have expiry dates.
- Environment rule changes, bypasses and out-of-band production changes raise alerts.
- Deployment records link to change tickets and are retained for the audit period.
Standards and references
- NIST: SP 800-53 Rev. 5 Security and Privacy Controls
- NIST: Cybersecurity Framework 2.0
- GitHub Docs: Deployments and environments
- GitHub Docs: Reviewing deployments
- GitHub Docs: Artifact attestations
- GitLab Docs: Deployment approvals
- Microsoft Learn: Define approvals and checks in Azure Pipelines
- MITRE ATT&CK: Compromise Software Supply Chain (T1195.002)
Related NHI Mgmt Group resources: DevSecOps Guide · CI/CD Security Guide · Segregation of Duties Guide · SLSA and Build Provenance Guide · Change Management Policy Template