SOPS and Sealed Secrets: Encrypt GitOps secrets and manage the keys safely

SOPS and Sealed Secrets let you commit encrypted secrets to Git, but the ciphertext lives forever in repository history, so the whole design rests on how well the decryption keys are scoped, backed up and replaced. Both tools answer the same GitOps question: how do you keep a declarative repository complete without putting plaintext credentials in it? They answer it differently. SOPS encrypts values inside YAML, JSON, ENV, INI or binary files against KMS, age or PGP recipients; Sealed Secrets encrypts a whole Kubernetes Secret to a public key whose private half only an in-cluster controller holds. This guide compares the two, explains the cryptography and scoping that matter, sets out key management and compromise response, and gives a rollout plan and checks. It is for platform engineers running Flux or Argo CD. The GitOps Security Guide gives the overview; this page goes deeper on these two tools.

Key takeaways

• Git history is permanent. Anyone who later obtains a decryption key can read every old revision, so a leaked key means rotating the real credentials, not just re-encrypting files.
• Sealed Secrets binds ciphertext to a name and namespace. The default strict scope uses the namespace and name as the RSA-OAEP label; cluster-wide removes that binding and lets a sealed value be unsealed anywhere.
• SOPS leaves structure readable. Key names and the sops metadata stay in cleartext for diffing; only matched values are encrypted, so set encrypted_regex deliberately.
• Key renewal is not secret rotation. The Sealed Secrets project says renewal and re-encryption are not a substitute for rotating the secret values; SOPS's rotate changes the data key, not your database password.
• Prefer cloud KMS identities to key files. A SOPS rule using KMS ties decryption to an IAM role you can audit and revoke; an age or PGP private key on a laptop cannot be audited at all.

How the two tools differ

• Choose Sealed Secrets when: secrets are only Kubernetes Secrets, developers should be able to encrypt but never decrypt, and you do not want a cloud KMS dependency.
• Choose SOPS when: the same encrypted file feeds several tools or clusters, reviewers need to see which keys changed, or you want decryption governed by cloud IAM.
• Choose neither when: you already run a central secrets manager and need frequent rotation. Referencing values with the External Secrets Operator security guide approach keeps ciphertext out of Git altogether.

Sealed Secrets: scopes, sealing keys and trust

• Sealing authenticates no one: the project notes that anyone can create a SealedSecret for a given namespace and name. What reaches the cluster must be controlled by repository review and Kubernetes RBAC on sealedsecrets and Secrets.
• Key renewal: the controller generates a new sealing key every 30 days by default (--key-renew-period; 0 disables it). Old keys are kept so existing SealedSecrets still decrypt, and they are not garbage collected.
• Back up every key: the documented backup is kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml. That file contains private keys: store it in a vault or HSM-backed store, refresh it after each renewal, and restrict who can read it.
• Use the right certificate: kubeseal --fetch-cert retrieves the public certificate for offline sealing with --cert. Distribute it from a trusted location so a developer is not tricked into sealing to an attacker's key.
• Validate before merge: kubeseal --validate in CI catches blobs sealed to the wrong cluster or scope.

SOPS: creation rules, recipients and integrity

SOPS behaviour is driven by .sops.yaml creation_rules. Review this file as carefully as code, because it decides who can decrypt every new file.

• Integrity: SOPS computes a MAC over all values and stores it encrypted under sops.mac, which is why editing a file needs the private key, not just the public one.
• Recipient changes: sops updatekeys adds or removes recipients from .sops.yaml without changing the data key; sops rotate generates a new data key and re-encrypts every value. After removing a recipient, rotate: the removed holder may already have the old data key.
• Key strength: SOPS does not enforce PGP key size. Its security notes recommend at least 2048-bit RSA or 256-bit ECDSA and keeping private keys on smart cards or offline.
• Flux decryption: set decryption.provider: sops on the Kustomization with a secretRef (for age the key entry must end .agekey), or bind an IAM role with KMS decrypt to the kustomize-controller service account in flux-system and hold no key file at all.

Where identity fits: the decryption key is a machine credential

The sealing key, the age identity and the KMS role are non-human identities with read access to every secret in the repository's history. Give each an owner, a scope (one per cluster or environment, never one key for all), a backup and a revocation plan. A KMS key used by SOPS should allow kms:Decrypt only to the in-cluster controller role and to a small break-glass group, with encryption allowed more widely. See the Secrets Management Guide and the Secrets Zero Problem Guide.

Key compromise and rotation

Assume any secret encrypted to a leaked key is exposed, including every historic revision. The two tools need different steps.

1. Scope the exposure: list every file or SealedSecret encrypted to the key across all branches and history.
2. Sealed Secrets: force a new sealing key early with --key-cutoff-time (RFC1123 time, as produced by date -R), mark the leaked key, rotate the real credentials, reseal them with the new certificate, then retire the old key once nothing depends on it.
3. SOPS: follow the documented order: remove the key from .sops.yaml, run sops updatekeys, run sops rotate, and only then rotate the underlying passwords and API keys. Doing it out of order can leave the compromised key with access.
4. Verify: confirm workloads use the new credentials and the old ones fail. See the Secrets Rotation Runbook Template.

Implementation plan

1. Find plaintext first: scan repositories, including history, for Kubernetes Secret manifests; scanners must decode base64. The exposed Kubernetes Secrets case involving SAP found registry credentials in committed Secret files that had been public for years.
2. Pick per use case: Sealed Secrets for cluster-only secrets with no KMS, SOPS with KMS where files cross tools or environments.
3. Pilot one non-production cluster: install the controller or Flux decryption, encrypt a test secret, confirm decryption and kubeseal --validate or a sops decrypt check in CI.
4. Add guardrails: pre-commit and CI checks that reject unencrypted Secret kinds, cluster-wide annotations without approval, and .sops.yaml changes without platform review (CODEOWNERS).
5. Migrate and rotate: move each secret, rotate the value (the old one was in plaintext), and remove plaintext from history where feasible.
6. Rollback: keep the previous controller version and backed-up sealing keys; restoring the keys restores decryption of existing blobs. For SOPS, older files remain decryptable as long as their recipients exist.

Detection and measurement

• Repository signals: commits adding kind: Secret outside approved paths, edits to .sops.yaml, and new scope annotations. See the Git Secret Scanning Guide.
• Cluster signals: Kubernetes audit events for reads of sealing key Secrets or the sops-age Secret, and SealedSecret unseal errors in controller logs.
• KMS signals: CloudTrail Decrypt calls on the SOPS key from any principal other than the controller role or break-glass users.

Framework mapping

Common failure patterns

• One age key for every environment: a single developer laptop compromise decrypts production and every past revision.
• Sealing keys lost with the cluster: no backup after renewal, so a rebuilt cluster cannot decrypt anything in Git.
• Re-encrypting instead of rotating: kubeseal --re-encrypt or sops rotate after a leak, while the actual credentials stay unchanged.
• Cluster-wide scope by default: chosen to make copying easy, it removes the protection that binds a value to its workload.
• Loose encrypted_regex: a rule that misses stringData or a config key commits secrets in cleartext beside encrypted ones.

Practitioner checklist

• Keep the Sealed Secrets strict scope unless an approved exception exists.
• Back up sealing keys after every renewal and test a restore at least quarterly.
• Use a separate sealing key or SOPS recipient set per cluster or environment.
• Prefer KMS recipients with role and encryption context over age or PGP key files.
• Set encrypted_regex: ^(data|stringData)$ for Kubernetes Secrets and keep mac_only_encrypted false.
• Protect .sops.yaml and controller configuration with CODEOWNERS review.
• Run kubeseal --validate or a decrypt check in CI, and block plaintext Secret kinds.
• On key compromise, follow the documented order and rotate the real credentials.

Standards and references

• SOPS: Reference (.sops.yaml and encryption)
• SOPS: Key management
• SOPS: Security
• SOPS: GitHub repository
• Bitnami Labs: Sealed Secrets README
• Bitnami Labs: Sealed Secrets cryptography
• Flux: Manage Kubernetes secrets with SOPS

Related NHI Mgmt Group resources: GitOps Security Guide · Kubernetes Secrets Security Guide · External Secrets Operator security guide · Git Secret Scanning Guide