How to check Kubernetes secrets encryption at rest: Verify etcd encryption on self-managed and managed clusters

A cluster can have an encryption setting switched on and still hold Secrets in etcd as plain text, because only Secrets written after the change are encrypted and a misordered provider list silently turns encryption off. The only way to know is to check the configuration, then check the stored data. This runbook shows how to do both on self-managed Kubernetes and how to read the equivalent status on Amazon EKS, Google Kubernetes Engine and Azure Kubernetes Service. It is for platform engineers, cluster administrators and auditors who need evidence, not assumptions. How to choose a provider and roll encryption out is covered in the Kubernetes Secrets Security Guide; this page is the check.

This is an independent guide from NHI Mgmt Group, based on the public documentation of the Kubernetes project, AWS, Google Cloud and Microsoft as of November 2026. It is not sponsored or reviewed by any of them. Managed service behaviour changes often, so confirm against current documentation.

Key takeaways

• The default is no encryption. Without --encryption-provider-config, or with identity as the first provider, the Kubernetes documentation states there is no confidentiality protection for Secrets in etcd.
• Configuration is not proof. Read a raw key from etcd and look for a prefix such as k8s:enc:kms:v2:; plain JSON or protobuf means that Secret was never rewritten.
• Provider choice matters as much as being on. The documentation rates aescbc as weak and not recommended, and aesgcm needs rotation every 200,000 writes; KMS v2 is the recommended target.
• Managed services report it differently. EKS encrypts all API data by default from Kubernetes 1.28, GKE reports databaseEncryption.state, and AKS reports a securityProfile block. Know which one you are reading.
• The key is now part of availability. Disabled or deleted KMS keys degrade or permanently break clusters on every major platform, so check key protection alongside encryption.

When you need this

• Audit or customer due diligence: controls such as NIST SP 800-53 SC-28 Protection of Information at Rest and CIS Control 3.11 Encrypt Sensitive Data at Rest need evidence per cluster.
• After enabling or rotating encryption: to confirm existing Secrets were rewritten and the old key is no longer needed.
• Before backup or disaster recovery changes: etcd snapshots carry whatever is stored, so plain text Secrets in etcd become plain text Secrets in backup storage. See Kubernetes Backup and Recovery.
• After a leak scare: the exposed Kubernetes Secrets incident involving an SAP artifact repository key shows how much access a single Secret can carry once it escapes the cluster.

Before you start

The documentation is explicit that a locally managed key protects against an etcd compromise but not a host compromise, because the key sits in the configuration file on the host. That is why the provider matters, not only the on or off state.

You need:

• Self-managed: shell access to a control plane node, read access to the API server manifest and encryption config, and etcd client certificates. These are cluster-admin equivalent: use a break-glass procedure and log the session.
• Managed: read-only cloud permissions to describe the cluster and the key (for example eks:DescribeCluster, container.clusters.get, or Reader on the AKS resource), plus key metadata read in the KMS.
• A test Secret: permission to create and delete one Secret in a scratch namespace, so you can verify new writes without touching real values.

Step-by-step: self-managed clusters

All steps are read-only except creating and deleting the test Secret.

1. Confirm the API server flag. On kubeadm-style clusters the API server is a static Pod. Inspect /etc/kubernetes/manifests/kube-apiserver.yaml on every control plane node for --encryption-provider-config=<path> and note whether --encryption-provider-config-automatic-reload=true is set. A missing flag on any node means Secrets written through that API server are stored unencrypted.
2. Read the provider order. Open the EncryptionConfiguration file. For the entry whose resources include secrets, the first provider encrypts new writes; the others are only tried when reading. Record: first provider type, key or plugin name, whether identity: {} is still present at the end, and whether a more specific entry (for example one exempting a resource) appears earlier and overrides it. Check that the file is identical on every control plane node.
3. Check KMS plugin health. For KMS providers, query the API server health endpoints. With KMS v2, or with automatic reload on, there is a single check at /healthz/kms-providers; otherwise each plugin has /healthz/kms-provider-0, /healthz/kms-provider-1 and so on, in config order:

   kubectl get --raw='/healthz/kms-providers'

4. Write and read a test Secret. Create a Secret named enc-test in a scratch namespace, then read it straight from etcd using the documented pattern:

   ETCDCTL_API=3 etcdctl \
      --cacert=/etc/kubernetes/pki/etcd/ca.crt \
      --cert=/etc/kubernetes/pki/etcd/server.crt \
      --key=/etc/kubernetes/pki/etcd/server.key \
      get /registry/secrets/<namespace>/enc-test | hexdump -C

   The stored value should begin with a provider prefix such as k8s:enc:kms:v2: followed by the plugin name, or k8s:enc:aescbc:v1: followed by the key name. The name in the prefix must match the config. Readable field names in the hex dump mean the write was not encrypted. The /registry prefix is the default; if your API server sets a different etcd prefix, adjust the path (the Kubernetes KMS page's own example uses a different one).
5. Sample existing Secrets. New writes passing does not prove old ones were rewritten. List keys under the Secrets prefix with etcdctl get /registry/secrets/ --prefix --keys-only, then read a sample from each namespace, prioritising old namespaces, kube-system and anything created before encryption was enabled. For a full check, script the same read for every key and count values without a k8s:enc: prefix. Never write the output to shared storage: it contains Secret values.
6. Check other sensitive resources. If teams store credentials in ConfigMaps or custom resources, check whether those resources are listed in the encryption config. Kubernetes only encrypts what the config names.
7. Clean up. Delete enc-test, clear shell history on the control plane node if it captured any values, and close the break-glass session.

Step-by-step: managed clusters

On managed services you cannot read etcd. The provider's reported state is your evidence, combined with key checks.

1. Record the platform state for every cluster with the commands above, from a read-only identity, and keep the output as audit evidence with the date.
2. Check the Kubernetes version against the default. On EKS the default only applies from 1.28. Older clusters with no encryptionConfig need an upgrade or a customer key; the EKS documentation states that Secrets encryption cannot be disabled once enabled on those versions.
3. Check that the existing data was re-encrypted. GKE re-encrypts existing Secrets when you enable the feature, which can take a long time. The legacy AKS KMS documentation tells you to rewrite Secrets with kubectl get secrets --all-namespaces -o json | kubectl replace -f - after enabling, rotating or disabling. Older EKS clusters used an annotate command to force re-encryption. Confirm the step was done and keep the change record.
4. Check the key. In the KMS, confirm the key is enabled, in the same region as the cluster where the platform requires it (EKS and GKE do), protected against deletion (deletion scheduling, soft delete and purge protection), and that only the platform's service identity has encrypt and decrypt rights. On GKE that identity is the Kubernetes Engine Service Agent with roles/cloudkms.cryptoKeyEncrypterDecrypter.

Verify it worked

• Self-managed: 100% of sampled etcd values carry the expected k8s:enc: prefix and key or plugin name, identity is absent from the Secrets provider list, and the KMS health endpoint returns ok.
• Managed: every cluster in the inventory reports the expected state, and the key in each report is the approved key.
• Read path: kubectl get secret enc-test -n <namespace> -o yaml returned the value through the API during the test, proving decryption works.

Rollback and common errors

This runbook only reads, so there is nothing to roll back except the test Secret. Fixing a failed check does change the cluster: follow the rollout and rollback steps in the Kubernetes Secrets Security Guide and back up etcd first.

Hardening afterwards

1. Protect the configuration and keys. Treat the EncryptionConfiguration file and the KMS key as being as sensitive as etcd itself. Alarm on key state changes, as the EKS documentation suggests with a CloudWatch alarm.
2. Turn on automatic reload for self-managed clusters so key rotation does not need an API server restart, and watch apiserver_encryption_config_controller_automatic_reload_last_timestamp_seconds to see when a new config took effect.
3. Encrypt etcd backups separately. Envelope encryption protects Secret values in a snapshot, but other resources may not be encrypted and the snapshot reveals names and structure.
4. Reduce what is in etcd. High-value credentials can be served from an external store so values never sit in etcd. See the Secrets Management Guide.
5. Repeat the check on a schedule and after every control plane upgrade, key rotation or restore.

Small and large estates

A team with one managed cluster can usually rely on the platform default, record the state quarterly and focus on key protection. Organisations with many clusters, or with self-managed control planes, should script the managed service checks into a nightly inventory job, alert on any cluster that is not in the target state, and run the etcd sample at least after each upgrade.

Metrics

• Encryption coverage: percentage of clusters reporting the target state (KMS v2, ENCRYPTED or better, or customer key where policy requires it).
• Stored plain text: count of sampled etcd Secret values without a k8s:enc: prefix. Target zero.
• Weak providers: clusters still using aescbc, aesgcm or KMS v1 as the first provider.
• Key protection: share of encryption keys with deletion protection and state alarms.
• Time since last verification per cluster.

Framework mapping

Practitioner checklist

• Confirm --encryption-provider-config on every self-managed control plane node.
• Record the first provider for Secrets and flag identity, aescbc, aesgcm and KMS v1.
• Check the KMS plugin health endpoint.
• Read a new test Secret from etcd and confirm the k8s:enc: prefix and key name.
• Sample existing Secrets across namespaces, starting with the oldest.
• Record EKS, GKE and AKS status with the platform commands and the Kubernetes version.
• Confirm existing Secrets were re-encrypted after enabling or rotating.
• Check key state, region, deletion protection and who can use the key.
• Repeat after upgrades, rotations and restores, and keep dated evidence.

Standards and references

• Kubernetes: Encrypting Confidential Data at Rest
• Kubernetes: Using a KMS provider for data encryption
• Kubernetes: Decrypt Confidential Data that is Already Encrypted at Rest
• AWS: Default envelope encryption for all Kubernetes API Data
• Google Cloud: Encrypt Secrets at the application layer
• Microsoft Learn: KMS data encryption in AKS
• Microsoft Learn: Add Key Management Service etcd encryption to an AKS cluster
• NIST: SP 800-53 Rev. 5

Related NHI Mgmt Group resources: Kubernetes Secrets Security Guide · Kubernetes Security Checklist · How to Find pods that automount service account tokens · Kubernetes Backup and Recovery