Ceph Security: Scope CephX Keys, Protect RGW and Harden the Dashboard

A Ceph cluster stores block volumes, file systems and S3 objects for many applications at once, and every one of them reaches that data with a CephX key or an RGW access key, so key scope and handling decide how far a single leaked credential can reach. Ceph gives operators fine-grained controls: capability strings per CephX entity, pool and namespace restrictions, encrypted msgr2 connections, RGW administrative capabilities and server-side encryption, and a dashboard with roles, SSO and an audit API. This guide explains how to use each of them, with exact commands and option names from the Ceph documentation, plus logging, a hardening baseline and a rollout order. It is written for storage, platform and security engineers running Ceph for private cloud, Kubernetes or object storage. Wider storage controls are covered in the Storage Infrastructure Security Guide.

This is an independent guide from NHI Mgmt Group, based on the Ceph project's public documentation as of November 2026. It is not sponsored or reviewed by the Ceph project. Product features change, so confirm settings against the Ceph project's current documentation.

Key takeaways

• Every CephX key is a machine identity. Give each application its own client. entity with capabilities scoped by profile, pool and namespace, never the client.admin key.
• Keep CephX on and turn on secure mode. The three auth_*_required options default to cephx; set ms_cluster_mode, ms_service_mode and ms_client_mode to secure to encrypt traffic with AES-GCM over msgr2.
• RGW admin and system users bypass IAM. The docs state that users with --admin or --system have global access that IAM policy cannot deny; disable their keys when not in use.
• Turn on the dashboard audit API. It is off by default; ceph dashboard set-audit-api-enabled true writes PUT, POST and DELETE requests to the Ceph audit log.
• Encrypt OSDs at deployment. encrypted: true in the cephadm OSD spec, optionally with tpm2: true, protects data on removed or failed drives.

How identity and access work in Ceph

Ceph has three identity planes, each with its own credentials:

CephX authentication

Three options in [global] control CephX: auth_cluster_required (daemon to daemon, default cephx), auth_service_required (clients to services, default cephx) and auth_client_required (cluster to client, default cephx, none). The documentation recommends against setting them to none, warning that without authentication any client can access the cluster. Monitors issue tickets: auth_mon_ticket_ttl defaults to 72 hours and auth_service_ticket_ttl to 1 hour.

Message signing is on where supported (cephx_sign_messages, default true). cephx_require_signatures, cephx_cluster_require_signatures and cephx_service_require_signatures default to false; set them to true once all clients support signatures. The docs note signatures give limited tamper protection and do not encrypt data in flight, which is why msgr2 secure mode matters.

Recent Ceph releases document aes256k as the preferred CephX key cipher and recommend moving keys from aes. Check the CephX configuration reference for your release before planning a key migration.

Encryption in transit with msgr2

Monitors listen on port 3300 for msgr2 and 6789 for legacy msgr1. msgr2 offers two modes: crc (authenticated, integrity-checked with crc32c, unencrypted) and secure (all post-authentication traffic encrypted with AES-GCM). Defaults prefer secure only for monitor connections (ms_mon_*_mode = secure crc), while ms_cluster_mode, ms_service_mode and ms_client_mode default to crc secure. To encrypt all traffic, set those three to secure, after running ceph mon enable-msgr2 on older clusters. Once every client uses msgr2, set ms_bind_msgr1 = false to close port 6789. Test client compatibility first, especially kernel RBD and CephFS clients, and benchmark CPU overhead.

Admin roles and privileged access

The client.admin key

/etc/ceph/ceph.client.admin.keyring is the default key for ceph commands and holds full rights. The documentation recommends copying it only to nodes that run administrative commands. Keyrings created in /etc/ceph by root are readable and writable by root only; keep it that way for admin keys. Our recommendation: keep the admin keyring on a small number of hardened admin hosts reached through your PAM workflow, rotate it with ceph auth rotate client.admin after staff changes, and use named admin entities per operator where your tooling allows so actions are attributable. See the Privileged Access Management Guide.

Capability design

Capabilities follow {daemon-type} '{cap-spec}'. Access specs are r, w, x, class-read, class-write and *. OSD caps can be limited with pool=, namespace=, object_prefix or tag {application} {key}={value}; without a pool limit, an OSD-capable user can reach every pool. Prefer built-in profiles:

Create users with ceph auth get-or-create client.app mon '...' osd '...' -o /etc/ceph/ceph.client.app.keyring. Remember that ceph auth caps overwrites all existing caps, so restate every cap when changing one. Remove users with ceph auth del, and review ceph auth ls regularly. See the NHI Ownership Guide for the ownership model.

Dashboard roles

Built-in roles are administrator, read-only, block-manager, rgw-manager, cluster-manager, pool-manager and cephfs-manager. Give most operators a scoped role, keep administrator to a few named accounts, and create users with the password in a file: ceph dashboard ac-user-create <user> -i <file> <role>, adding --pwd_update_required for first-login change. Avoid --force-password, which skips policy checks.

Integrations, API tokens and service accounts

RGW S3 keys

radosgw-admin user create generates an S3 access key and secret key. Treat each as a machine credential:

• One user per application: separate users give separate keys, quotas, rate limits and usage records.
• Rotation: add a new key with radosgw-admin key create --uid=<uid> --key-type=s3 --gen-access-key --gen-secret, move the application, then remove the old one with key rm. Our recommendation: rotate application keys at least every 90 days and immediately on exposure.
• Subusers: Swift subuser access levels are read, write, readwrite and full; full also manages access control policies, so use readwrite for applications.
• Suspension: radosgw-admin user suspend disables a user and its subusers; bucket suspend returns 403 BucketSuspended for one bucket.
• Admin capabilities: users have no Admin Ops API access by default. Grant only the needed caps (users, buckets, metadata, usage, zone, ratelimit) and add user-info-without-keys to tooling so user info output does not include secret keys.
• Admin and system users: --admin and --system grant global read and write that IAM cannot deny. Use --system only for multisite as documented, and disable admin keys when not in use.

Store RGW keys in a secrets manager and inject them at run time. See the Secrets Management Guide.

Server-side encryption in RGW

Encryption requests must use HTTPS. If a proxy terminates TLS, set rgw trust forwarded https. For new deployments the docs recommend rgw crypt sse algorithm = aes-256-gcm once all RGW instances support it. rgw crypt default encryption key is documented for diagnostics only, and the docs note the Ceph configuration file is not a safe place to store keys. See the HashiCorp Vault Guide and the Envelope Encryption and KMS Integration Patterns Guide.

Other integration credentials

• Dashboard to RGW: cephadm configures this automatically; manually, ceph dashboard set-rgw-credentials creates an RGW user with uid dashboard. Include it in key reviews.
• Kubernetes CSI and OpenStack: give each consumer its own CephX entity with profile rbd on its own pool, stored as a Kubernetes secret or service configuration with restricted access.

Logging and detection

• Cluster audit log: monitor commands, including ceph auth changes, are recorded in the Ceph audit channel. Forward it to the SIEM.
• Dashboard audit API: ceph dashboard set-audit-api-enabled true logs PUT, POST and DELETE with origin, path, method and user. Payload logging is on by default; set set-audit-api-log-payload false if payloads could include passwords or keys.
• RGW usage log: rgw_enable_usage_log = true records per-user usage for anomaly detection. Set rgw_usage_max_shards consistently across daemons.
• RGW access logs: forward gateway or front-end proxy logs, keyed by access key ID, for request-level detection.

Metrics: count of CephX entities with OSD caps not limited to a pool (target zero beyond daemons); hosts holding the admin keyring; percentage of connections in secure mode; RGW keys older than your rotation period; RGW users with admin or system flags; dashboard accounts with administrator.

Hardening baseline

Dashboard

• TLS: on by default. Replace the self-signed certificate with ceph dashboard set-ssl-certificate and set-ssl-certificate-key, using an RSA key as the docs specify, then fail over the manager to load it. Keep mgr/dashboard/ssl at true.
• Binding: the default address is :: on ports 8443 (TLS) or 8080. Set mgr/dashboard/server_addr to a management interface and restrict it with host firewalls. See the Linux Firewall Guide.
• Authentication: configure SSO with ceph dashboard sso setup saml2 or OAuth2 through the cephadm management gateway, so MFA and leaver processes come from your identity provider. Keep one break-glass local administrator.
• Password policy and lockout: both are on by default (minimum length 8, minimum complexity 10). Raise the minimum length, keep lockout on with set-account-lockout-attempts, and never set it to 0.
• Debug mode: keep it off; it adds tracebacks to responses.

Cluster and storage

• Keep auth_*_required = cephx; set the cephx_*_require_signatures options to true once clients support them.
• Set msgr2 secure mode for cluster, service and client traffic, then disable msgr1 binding.
• Separate public and cluster networks and restrict ports 3300, 6789 and OSD ports to known hosts. See the Network Segmentation Guide.
• Deploy OSDs with encrypted: true; add tpm2: true (Squid onward, LUKS2) where hosts have TPM2. method: raw encryption requires TPM2; use method: lvm otherwise. See the Database and Storage Encryption Guide.
• Set RGW quotas and rate limits per user, remembering both are tracked per gateway instance.
• Run a release in Ceph's Active Releases list, which is where security updates land, and follow the Ceph security page.

Rollout order

1. Export ceph auth ls and radosgw-admin user list; map every entity and RGW user to an owner and application.
2. Create scoped entities per application, move applications one by one, then remove shared and unused keys.
3. Remove the admin keyring from non-admin hosts and route admin work through hardened hosts.
4. Enable the dashboard audit API, SSO and a CA-signed certificate; restrict its listener.
5. Test msgr2 secure mode in staging with every client type, then enable it in production; keep the previous mode values ready to restore as rollback.
6. Set SSE defaults on buckets with a KMS backend and use encrypted OSDs for new and replaced drives.

Common misconfigurations

• The admin keyring on every node and in automation. Fix: scoped entities per tool, admin key on admin hosts only.
• OSD caps without a pool limit. Fix: pool= and namespace= on every client entity.
• ceph auth caps used to add one cap. It replaces the full set and can break or widen access. Fix: restate every cap, test in staging.
• Admin flag on an application's RGW user. Fix: specific admin caps, with admin users disabled when idle.
• Encryption keys in ceph.conf. Fix: SSE-KMS or SSE-S3 with a KMS.
• Dashboard on all interfaces with local passwords only. Fix: management network binding, SSO, audit API.

Practitioner checklist

• Confirm auth_cluster_required and auth_service_required are cephx on every cluster.
• Give each application its own CephX entity with profile, pool and namespace limits; delete unused entities.
• Keep ceph.client.admin.keyring on hardened admin hosts only and rotate it after staff changes.
• Enable msgr2 secure mode for cluster, service and client traffic, then stop binding msgr1.
• Issue one RGW user per application, rotate S3 keys on a schedule, and remove admin or system flags not strictly needed.
• Use SSE-KMS or SSE-S3 with a KMS for buckets holding sensitive data; never keep keys in configuration files.
• Bind the dashboard to a management network, replace the self-signed certificate and turn on SSO.
• Enable the dashboard audit API and RGW usage log, and forward the Ceph audit channel to the SIEM.
• Deploy OSDs with encrypted: true, using TPM2 where available.
• Stay on an Active Release and watch the Ceph security page for updates.

Standards and references

• Ceph: User Management
• Ceph: CephX Config Reference
• Ceph: Messenger v2
• Ceph: Ceph Dashboard
• Ceph: Object Gateway Admin Guide
• Ceph: Object Gateway Server-Side Encryption
• Ceph: cephadm OSD Service
• Ceph: Security

Related NHI Mgmt Group resources: Storage Infrastructure Security Guide · Database and Storage Encryption Guide · Secrets Management Guide · Kubernetes Backup and Recovery