Terraform is not a recovery plan for cloud infrastructure incidents

At a glance

What this is: Commvault argues that Terraform is for provisioning desired state, while recovery depends on snapshots of actual deployed cloud state.

Why it matters: This matters because teams that confuse infrastructure-as-code with recovery tooling may be unable to restore cloud services after deletion, drift, or failed changes.

Context

Terraform is a cloud provisioning and change-management tool, not a recovery system. The governance gap is assuming that code, state files, and Git history are enough to reconstruct what was actually running when an incident hits.

That distinction becomes operationally important when infrastructure changes arrive through the console, manual intervention, or drift outside the IaC workflow. For IAM and NHI teams, the same lesson applies to any control plane: the source of truth for creating change is not automatically the source of truth for restoring service.

Key questions

Q: What breaks when Terraform is treated as a recovery tool?

A: Teams can redeploy desired configuration, but they may not be able to reconstruct the exact environment that was running before the incident. That becomes a serious problem when console edits, manual fixes, or drift changed production outside the code path. Recovery then depends on incomplete records instead of actual state.

Q: Why do code, state files, and Git history fall short for cloud recovery?

A: They capture configuration intent and version history, not a point-in-time snapshot of what was running. If a change happened through the console, manually, or outside the IaC pipeline, those artifacts may miss the very difference that caused the outage. Recovery needs live-state evidence, not just source control and state metadata.

Q: How should teams design recovery when cloud drift is possible?

A: They should treat drift as a normal operating condition and maintain a separate restoration path for actual deployed state. That means testing whether the environment can be rewound to a specific time, even when Terraform state is incomplete or stale. The goal is to recover service, not merely re-run deployment logic.

Q: When should organisations rely on point-in-time recovery instead of infrastructure-as-code rollback?

A: Use point-in-time recovery when the incident changed live resources, introduced manual edits, or deleted infrastructure that Terraform no longer accurately represents. Infrastructure-as-code rollback is useful for controlled change reversal, but it is not a substitute for restoring the deployed environment that existed before the failure.

Technical breakdown

Desired state versus actual deployed state

Terraform applies a declared configuration to produce the intended cloud environment. That makes it a desired-state system, where the codebase represents what should exist, not a historical record of every live resource and setting. Recovery tools work differently: they preserve snapshots of the deployed environment so the team can return to a known-good point in time after change, deletion, or corruption. The core issue is that desired state can be re-applied, but actual state must be captured if you want to restore what was truly running.

Practical implication: Use provisioning and recovery as separate control functions, and do not assume infrastructure code can reconstruct a live incident state.

Why Terraform state and Git history are not recovery artifacts

Terraform state files describe the current or last-known relationship between code and resources, while Git records versioned configuration intent. Neither one is a point-in-time snapshot of the running environment. If a change was made outside Terraform, or if drift occurred after the last apply, those artifacts may be incomplete or misleading. In incident recovery, the difference between configuration history and deployed history matters because a clean code rollback can still leave the service in the wrong runtime state.

Practical implication: Treat state files and Git as change-management inputs, not as authoritative recovery evidence.

Point-in-time cloud recovery and out-of-band change handling

Point-in-time recovery depends on capturing actual cloud configuration so it can be rewound to a specific moment. That is what closes the gap left by console changes, manual edits, accidental deletion, and infrastructure drift. The mechanism is backward-looking, while Terraform is forward-looking. In practice, this means the recovery layer must preserve the live environment separately from the provisioning workflow, otherwise the team is forced to rebuild from assumptions instead of restoring the thing that failed.

Practical implication: Maintain a recovery mechanism that records actual cloud state independently of infrastructure-as-code pipelines.

NHI Mgmt Group analysis

Recovery requires an operational state record, not just configuration code. Terraform proves that cloud change can be repeatable, but repeatability is not the same as recoverability. When a production incident unfolds, teams need a record of what was actually deployed, not only what was intended. The practitioner lesson is to separate infrastructure provisioning from infrastructure restoration in governance and tooling.

The real failure mode is state drift plus false confidence in version control. Manual console edits, out-of-band changes, and deleted resources create a recovery gap that code history cannot close. That gap is especially relevant in cloud environments where human, automated, and NHI-driven changes can all touch the same control plane. The practitioner conclusion is to treat live-state capture as a distinct resilience control.

Actual-state recovery is becoming a cloud resilience baseline. The more distributed and fast-moving the environment, the less reliable it is to assume a rebuild from code will match the pre-incident runtime. This is a governance issue as much as a technical one, because incident readiness now depends on whether teams can prove they can restore the environment they had, not the one they planned. Practitioners should test restoration against the real deployed state, not the repository.

Terraform and recovery tooling are complementary because they solve different parts of the lifecycle. One governs creation and change, the other governs restoration and continuity. That split should inform RTO planning, backup strategy, and cloud operating model design. The practitioner conclusion is to map each control to its actual job instead of relying on IaC to cover both build and recovery.

What this signals

Actual-state recovery is a resilience control, not a convenience feature. Cloud teams increasingly need to prove that they can restore the environment that was really running, including changes introduced outside the IaC workflow. That requirement affects incident readiness, backup design, and how you define the boundary between provisioning and recovery.

For identity and access programmes, the same operating principle applies to cloud control planes and privileged change paths. If your governance model only records intended state, you will miss the evidence needed to restore service after drift, deletion, or manual intervention.

For practitioners

• Separate provisioning from recovery controls Document Terraform as the desired-state engine and designate a distinct recovery process for restoring live cloud environments after incidents.
• Test restoration from actual deployed state Run incident drills that restore a known-good cloud snapshot rather than replaying code and state files, so teams validate true recovery.
• Inventory out-of-band change paths Identify console edits, manual interventions, and drift sources that can bypass Terraform so recovery assumptions do not depend on incomplete state.
• Align recovery evidence with operational reality Confirm that your incident response plan can prove what was running at a point in time, especially for environments with frequent non-IaC change.

Key takeaways

• Terraform is built to manage desired state, but desired state alone does not equal recoverable state.
• Point-in-time recovery depends on capturing the deployed environment as it actually existed before the incident.
• Teams should separate infrastructure provisioning from recovery so cloud restoration can succeed even when changes bypass IaC.

Key terms

• Desired state: The target configuration or operating condition an administrator wants a device to reach and maintain. In declarative systems, desired state replaces many direct commands with policy, rules, and configuration data. The practical challenge is ensuring the declaration is accurate, complete, and auditable across the fleet.
• Actual Deployed State: The live configuration and runtime condition of cloud infrastructure at a specific moment. Recovery depends on this state because it captures what was truly operating, including changes introduced outside code, state files, or normal deployment workflows.
• Infrastructure Drift: Infrastructure drift is the gap between the configuration a team thinks is deployed and the state that actually exists in cloud. In identity terms, drift weakens governance because policy, ownership, and remediation no longer map cleanly to live assets.
• Point-in-Time Recovery: Point-in-time recovery restores data or assets to the exact state they had at a specific moment before deletion, corruption, or unwanted change. In SaaS analytics environments, it must preserve the object and the dependencies needed for it to function, not only a static export.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, secrets management, and the lifecycle controls that underpin resilient access models. It is a practical fit for practitioners who need stronger identity governance across cloud and automation-heavy environments.