Active-Active vs Active-Passive: How to Choose Failover for Identity and Secrets Systems

Teams often pick active-active because it sounds safer, then discover that the same replication that removes downtime also copies a compromise, a corrupt write or a ransomware deletion to every site within seconds. Active-active and active-passive are two ways of building high availability: in the first, every site serves live traffic; in the second, one site serves traffic while another waits to take over. For ordinary web tiers the choice is mostly about cost and recovery time. For security-critical systems such as an identity provider (IdP), a privileged access management (PAM) vault or a secrets manager, it also decides how sessions, tokens and key material behave during a failure, and how far an attacker's changes travel. This guide compares the two designs as a security decision: failure modes, recovery objectives and cost, state and key handling, where active-active amplifies compromise, and decision criteria for each class of tier 0 system. It is written for architects and owners of identity and secrets platforms. For secrets manager clustering detail see the Secrets Manager Resilience Guide; this page goes deeper on the choice itself.

Key takeaways

• Availability is not recoverability. Both designs protect against a site or region failing; neither protects against bad data, because replication faithfully copies deletions, corruption and malicious configuration changes to the other side.
• Active-active trades failover risk for consistency risk. There is no failover step to go wrong, but concurrent writes, conflict resolution and split traffic make state, revocation and audit harder to reason about.
• Active-passive fails where it is not exercised. A standby that is never promoted in a planned switchover is a hope, not a control, and split brain after an unfenced failover can lose data.
• Tier 0 systems need a third, isolated copy. Point-in-time backups in a separate account or tenant, with separate credentials, are what recover you from a compromise that both live sites share.
• Choose per system, not per estate. An IdP, a PAM vault and a secrets manager have different write patterns and different tolerance for stale data, so the right answer often differs between them.

The short answer and the comparison table

Choose active-passive with a tested, fenced promotion when strong consistency of security state matters more than seconds of downtime, which is the common case for PAM vaults and for the write path of secrets managers. Choose active-active when the workload is read-heavy, latency-sensitive and tolerant of brief staleness, such as token validation, federation endpoints and read replicas of secrets, and when you can partition or forward writes so that two sites never accept conflicting changes to the same object. In both cases keep an isolated backup copy that neither live site can modify.

How each design works

Active-active

Two or more complete deployments run at once and a global routing layer spreads users between them, usually by latency or weight. Microsoft's Well-Architected guidance describes two variants: at capacity, where each site carries its own load and scales up to absorb another site's load during an outage, and overprovisioned, where each site is already sized for the extra load. The at-capacity variant is cheaper but users may see degraded service while it scales. Capacity is easy to get wrong: Microsoft's own example is two nodes each running at 60 per cent, which leaves the survivor at 120 per cent when one fails.

The hard part is data. AWS describes three write patterns for multi-site designs:

• Write global: all writes go to one site and another site is promoted if it fails. Reads are local. This is active-active for reads and active-passive for writes, and it is the pattern most security systems should use.
• Write local: each site accepts writes and the data layer reconciles. Amazon DynamoDB global tables, for example, resolve concurrent updates with "last writer wins".
• Write partitioned: each record has a home site by key, such as user ID, so two sites never write the same object.

Active-passive

The primary carries all traffic and replicates to a standby. A warm standby runs at reduced size and only needs scaling up; a cold standby has resources defined but stopped, so it must be started before it can serve. AWS adds a pilot light tier where only the data layer runs. Failover needs three things that the database usually does not provide on its own: failure detection, a promotion decision and a way to stop the old primary. The PostgreSQL documentation is explicit that it does not include the software to detect a primary failure and notify the standby, and that if the old primary comes back without being told it has been demoted, "both systems think they are the primary", which leads to data loss. Its answer is fencing, which it names STONITH (shoot the other node in the head).

Failure modes compared

The quorum arithmetic matters for both designs. HashiCorp documents that a Raft peer set needs a majority, so three nodes tolerate one failure, five tolerate two, and a two-node set tolerates none; if a cluster cannot reach quorum it becomes unavailable and cannot commit writes. Stretching three nodes across two data centres therefore means losing the wrong site stops the cluster. Put the tie-breaking node in a third location or accept manual recovery.

RTO, RPO and cost

Recovery time objective (RTO) is how long the service may be down; recovery point objective (RPO) is how much recent data you may lose. Set both from a business impact analysis, as NIST SP 800-34 Rev. 1 describes for contingency planning, not from what a product brochure promises. Then set them twice: once for infrastructure loss and once for a data disaster. AWS states plainly that for active-active, recovery from corruption, deletion or obfuscation will always take longer than zero, with a recovery point before the problem was discovered.

Every pattern ends at the same place when the problem is bad data. A second live site buys protection against outages, not against attackers with administrator rights, so budget the isolated backup first.

How each handles sessions, tokens and key material

• User sessions: in active-active, a session created in one site must be readable in the other or users are logged out when routing shifts. Keycloak's multi-cluster design, for example, stores user sessions, authentication sessions and tokens in a synchronously replicated database, and its documentation notes this roughly doubles database CPU and write load. Sticky routing hides the problem until a site fails.
• Tokens and leases: HashiCorp Vault shows the two models side by side. Disaster recovery secondaries mirror tokens and leases but do not serve requests until promoted, so clients keep working after promotion. Performance secondaries serve reads locally but keep their own tokens and leases, so after promotion applications must reauthenticate. When disconnected, a performance secondary still answers reads, which may be stale, while writes fail.
• Revocation: a revoked token or disabled account is only safe once every site that validates it has seen the change. Keep revocation and credential changes on a single write path and keep access token lifetimes short so the stale window is bounded. See the Access Tokens vs Refresh Tokens.
• Key material: signing keys, encryption keys and hardware security module (HSM) partitions must exist in every site that signs or decrypts. Replicating them widens exposure; not replicating them turns a site loss into a key loss. Azure Key Vault illustrates a managed passive model: contents replicate asynchronously to the paired region, Microsoft initiates failover on a best-effort basis that can take hours, and the secondary is read-only, allowing get, list, encrypt, decrypt, wrap, unwrap, sign, verify and backup but no configuration changes. See the Cryptographic Key Management Guide.
• Audit trail: active-active scatters events for one user across sites. Forward all sites to one log platform with synchronised clocks, or investigators cannot reconstruct the sequence.

Where active-active amplifies compromise

Active-active removes the human pause that active-passive often has. An attacker who gains administrative access to an IdP or secrets manager in one site can create a federation trust, add an administrator, issue long-lived credentials or delete a secrets engine, and the change is live in every site immediately, with no promotion step at which someone might notice. Microsoft's Active Directory forest recovery guidance lists exactly these cases as forest-wide failures: a rogue administrator, a script that spreads corruption across the forest, malicious schema changes and malware on domain controllers. Multi-master replication makes Active Directory the classic example, and it is why forest recovery starts from backup, not from a surviving replica.

Three design choices reduce the blast radius:

1. Separate the control plane from the data plane. Let both sites serve authentication and secret reads, but route configuration writes, role changes and key operations to one site through a privileged path.
2. Isolate the recovery copy. AWS recommends a separate account per Region and cross-account backup copies to protect against account compromise and insiders. CISA's ransomware guide recommends offline, encrypted backups because ransomware looks for and destroys accessible ones. See the Backup Identity Isolation Guide.
3. Keep a delayed or versioned copy. Object versioning and delete markers that do not replicate let you step back past a malicious deletion. AWS notes that with replication configured this way, a delete in the source adds a marker only in the source bucket.

Decision criteria for IdP, PAM vault and secrets manager

Small organisations: a SaaS IdP and a managed secrets service already carry multi-site resilience. Spend effort on break-glass access, configuration backup and client retry behaviour rather than building your own second site. Large organisations: self-hosted tier 0 platforms usually justify read-local, write-global designs, with a DR replica in a third failure domain and a cyber recovery copy in an isolated account.

Worked scenario: a self-hosted secrets manager across two regions

A retailer runs a five-node secrets manager cluster in one region serving 400 services in two regions. Cross-region latency adds 80 ms to every secret read, so the platform team proposes full active-active. The review reaches a different design:

1. Reads go local: a read replica cluster in the second region serves static secrets and encryption operations. Writes and dynamic credential issuance are forwarded to the primary.
2. Whole-region loss goes to DR: a separate DR replica in a third location mirrors tokens and leases, so promotion does not force every service to reauthenticate at once.
3. Promotion is manual and fenced: on-call engineers promote only after confirming the primary is unreachable from two vantage points and its load balancer target is removed. A planned switchover runs every quarter.
4. Compromise recovery is separate: hourly snapshots copy to a backup account whose administrators hold no role in the production account, and recovery keys are held offline under dual control.
5. Clients are hardened: services cache secrets within their lease, retry with exponential backoff and alert when they fall back to cache.

The result keeps local reads, avoids conflicting lease writes and gives a recovery path that trusts neither live region.

Implementation and testing

1. Classify state. For each system list which objects are security-relevant writes (credentials, roles, policies, keys, revocations) and which are tolerant reads.
2. Set two sets of objectives. RTO and RPO for site loss, and RTO and RPO for corruption or compromise, approved by the service owner.
3. Design the write path. Prefer write global or write partitioned. Accept last-writer-wins only for data where losing a concurrent update is harmless.
4. Build fencing and quorum. Odd-numbered voting members, a third failure domain for the tie-breaker, and a mechanism that stops a demoted primary from accepting writes.
5. Codify both sites. Deploy both from the same infrastructure as code, as CISA recommends for rebuilds, and check for drift weekly.
6. Test in rings. Start with a planned switchover in a non-production environment, then production during a maintenance window, then an unannounced exercise. PostgreSQL's documentation recommends regular planned switchovers because they also test the failover mechanism.
7. Test the data disaster. Restore a tier 0 system from the isolated backup into a clean network, rotate its keys and confirm clients reconnect. See the Disaster Recovery Testing Guide.
8. Define rollback. Document failback: rebuild the old primary as a standby from the new primary rather than letting it rejoin with divergent data.

Detection and measurement

• Replication lag per replica, alerting at a threshold below your RPO.
• Quorum and leader changes: unexpected elections, voter loss and nodes that rejoin as primary.
• Conflict counts in multi-write stores, and the object types involved.
• Administrative changes on any site, especially trust, role and key operations, forwarded to the SOC in real time.
• Backup integrity and the success of restores in the isolated account.

Metrics that show the design is working:

• Measured failover or switchover time against RTO, per quarter.
• Maximum observed replication lag against RPO.
• Percentage of tier 0 systems with a successful isolated restore in the last six months.
• Days since the last planned switchover for each active-passive pair.
• Time for a revocation to become effective in every site.

Common failure patterns

• Calling replication a backup. Both live sites hold the deleted vault or the malicious trust; there is nothing clean to fail to.
• Backups administered by the same identities. An attacker with tier 0 rights deletes production and recovery copies together. Keep backup administration separate, as covered in the Backup Security and Immutable Backup Guide.
• Automatic promotion without fencing. A brief network blip creates two primaries and divergent credentials.
• A standby that has never taken traffic. Certificates, firewall rules and local configuration are found missing during the real outage.
• Break-glass that depends on the failed system. Emergency accounts stored in the PAM vault or authenticated by the IdP that is down. See the Break-Glass Account Guide.

Framework mapping

Practitioner checklist

• Separate RTO and RPO targets exist for site loss and for corruption or compromise on every tier 0 system.
• Security-relevant writes (credentials, roles, keys, revocations) go through a single write path or a partitioned design.
• Voting clusters use odd numbers with the tie-breaker in a third failure domain.
• Promotion includes fencing of the old primary, and tier 0 promotion is a deliberate human decision.
• A planned switchover runs at least quarterly for each active-passive pair.
• Point-in-time backups sit in a separate account or tenant with separate administrators and an offline or immutable copy.
• A restore of each tier 0 system into a clean environment has been tested within six months.
• Break-glass access works when the IdP and PAM vault are both unavailable.
• Logs from all sites reach one platform with synchronised time.

Standards and references

• NIST: SP 800-34 Rev. 1, Contingency Planning Guide for Federal Information Systems
• CISA: #StopRansomware Guide
• AWS: Disaster recovery options in the cloud
• Microsoft: Highly available multi-region design strategies
• Microsoft: AD forest recovery, identify the problem
• PostgreSQL: Failover
• HashiCorp: Vault replication
• Keycloak: High availability introduction

Related NHI Mgmt Group resources: Business Continuity and Disaster Recovery Guide, Cloud Region Outage Resilience, Privileged Access Management Guide, Windows Failover Cluster Security.