Configuration as code records resources that are represented in Git and Terraform state, so recovery depends on code accurately matching the live environment. Backup and recovery protects the operational system itself by capturing live PagerDuty configuration over time and preserving known-good recovery points, including changes made outside the IaC workflow.
Configuration as Code vs Backup: What Changes in Practice
configuration as code and backup solve different failure modes, even when both are used for the same PagerDuty environment. Code is strongest when the desired state is fully represented in Git and Terraform state, because you can recreate or reconcile from source-of-truth definitions. Backup is stronger when you need to preserve the live operational system, including edits made outside the IaC workflow, and restore from a point in time that already worked.
The practical difference is that code assumes the repository and state are accurate enough to reconstruct the service, while backup assumes the running configuration itself is the thing worth preserving. That means code is usually the better fit for repeatability, review, and controlled change, while backup is the better fit for recovery from drift, accidental edits, and changes that never made it into version control.
Where the Two Approaches Overlap and Where They Do Not
They overlap only partially because they protect different assets. Configuration as code helps you detect and reapply intended configuration, but it does not by itself guarantee that the current PagerDuty settings match the real operational state. Backup captures the actual state at a given time, which is useful when the live environment has diverged from the repository or when a manual change was made under pressure and later needs to be restored.
That distinction matters most during incident recovery and change rollback. If a team only keeps Terraform or Git history, the recovery path depends on code quality and drift control. If a team only keeps backups, the recovery path may restore a state that is safe but not reproducible or reviewable. In practice, the best operating model is usually to treat code as the control plane for intended change and backup as the safety net for restoring the service itself.
For teams managing broader identity and secret sprawl alongside configuration state, NHIMG’s Ultimate Guide to Non-Human Identities is a useful reference point for why drift, visibility, and lifecycle control matter across operational systems. The same logic applies to PagerDuty configuration, even though the object being protected here is the configuration state rather than credentials alone.
Risk and Threat Considerations
Configuration drift is the main risk when you rely on code alone, especially if admins can still make live changes in PagerDuty outside the IaC workflow. The operational exposure is not just inconsistency, but the possibility that a rollback, audit, or re-provisioning event silently drops unmanaged changes or reintroduces an old setting that no longer reflects how the service is actually used.
Failure mechanism: The repository or state file becomes stale, incomplete, or out of sync with live PagerDuty settings, so recovery from code reproduces the wrong configuration or misses important manual changes. A backup model reduces that gap by preserving known-good live states over time.
Impact: Teams can restore service faster after a bad change, but they also need to manage backup freshness, retention, and the possibility that a restored configuration reflects an older operating assumption. If the change was security-sensitive, stale restoration can re-enable legacy routing, permissions, or notification logic that should have been retired.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 address the attack and risk surface, while CIS Controls v8 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| CIS Controls v8 | CIS Control 4 — Secure Configuration of Enterprise Assets and Software | PagerDuty configuration management is a secure configuration problem with drift and recovery implications. |
| CIS Control 10 — Data Recovery | Backup and recovery of live configuration is a recovery control, not just source control. | |
| Recommendation — Record approved PagerDuty settings and restore known-good configurations from validated baselines. Back up live PagerDuty configuration and test restores to known-good points in time. | ||
| NIST CSF 2.0 | PR.IP-1 — Baseline Configuration | Managing PagerDuty as code depends on a maintained baseline that can be compared to live state. |
| RC.RP-1 — Recovery Plan Executed | Backup-focused recovery requires a tested process for restoring operational configuration after change failure. | |
| Recommendation — Define and maintain a baseline for PagerDuty configuration and compare live state against it. Test PagerDuty restore procedures so recovered configuration is usable during an incident. | ||
| OWASP Non-Human Identity Top 10 | NHI-03 — Secrets and Credential Management | Configuration drift and live-state backup concerns often intersect with operational secret handling in automation workflows. |
| Recommendation — Protect any secrets embedded in PagerDuty automation and keep recovery copies aligned with current rotation state. | ||
Practitioner Guidance
What to verify: Confirm which PagerDuty objects are actually governed by Terraform and which can still be modified directly in the UI or API. If a setting is both security-sensitive and likely to drift, treat backup coverage as mandatory rather than optional.
Decision rule: Use configuration as code when you want controlled change, reviewability, and reproducibility. Use backup when you need point-in-time recovery from live state, especially for changes made outside the IaC pipeline or during urgent operational work.
Practitioner takeaway: The safest model is not “code or backup”, it is code for intended state and backup for restoration of the real state when the two inevitably diverge.
Related resources from NHI Mgmt Group
- What is the difference between backing up observability infrastructure and backing up the applications it monitors?
- What is the difference between managing MongoDB Atlas manually and managing it with infrastructure-as-code workflows?
- What is the difference between managing AWS CodeBuild manually and managing it with infrastructure as code?
- What is the difference between managing Terraform code in Git and controlling who can deploy it to cloud infrastructure?
