Elasticsearch API keys are bearer credentials that carry a snapshot of their owner's privileges, and unless you set an expiry when you create them through the API they stay valid until someone invalidates them. They end up in Beats and Elastic Agent configs, Logstash pipelines, application settings, CI variables and remote cluster keystores, often created by a person who later changes role or leaves. This guide covers the lifecycle of Elastic API keys: the three key types, how role descriptors scope a key, expiry and rotation, cross-cluster API keys and their strong identity verification option, inventory with the query API, audit events, and leak response. It is written for platform, search and security engineers who administer Elasticsearch, Kibana or Elastic Cloud deployments, and it goes deeper on keys than the Elastic Security Guide.
This is an independent guide from NHI Mgmt Group, based on Elastic's public documentation as of November 2026. It is not sponsored or reviewed by Elastic. Product features change, so confirm settings against Elastic's current documentation.
Key takeaways
- Always pass role descriptors. A REST API key created without
role_descriptorsreceives a point-in-time snapshot of everything its owner can do; with descriptors, its access is the intersection of the descriptors and the owner's privileges. - Set an expiry every time. Keys created through the create, grant and cross-cluster APIs have no expiry unless you set
expiration; the Kibana simple creation flow on Elastic Cloud Hosted and ECE applies 90 days. - Treat cross-cluster keys as a separate class. They are created only with
manage_security, grant exactly what theaccessobject lists, cannot authenticate REST calls, and can be bound to a signing certificate withcertificate_identity. - Owner changes reach the key. Updating a key refreshes the owner's privilege snapshot, and sending an empty
role_descriptorsobject removes the restriction so the key inherits the owner's full access. - Inventory with the query API, alert on the audit trail. The query API key information API lists every key with owner, realm, creation and expiry, and the
create_apikey,change_apikeyandinvalidate_apikeysaudit events show who changed what once actor attribution is turned on.
The three Elastic API key types
Elastic documents three kinds of API key. They share the API key service (enabled by default through xpack.security.authc.api_key.enabled) but differ in who creates them, what they can do and which privilege manages them.
| Type | Purpose | Created with | Privilege to create or update | How access is set |
|---|---|---|---|---|
| User (REST) API key | External services call Elasticsearch and Kibana APIs on behalf of a user | POST /_security/api_key, grant API key API, Kibana API keys page | manage_own_api_key (own keys) or manage_api_key | Intersection of role_descriptors and the owner's privileges, or the owner's full snapshot if none are given |
| Cross-cluster API key | Another cluster connects for cross-cluster search or replication | POST /_security/cross_cluster/api_key, Kibana API keys page | manage_security; Kibana states an Enterprise licence is needed | Exactly the access.search and access.replication entries; the creator's privileges are not captured |
| Managed API key | Kibana runs background tasks such as rules and alerting | Created and managed by Kibana | Managed by Kibana | Snapshot of the user who created or last edited the object |
The managed keys behind Kibana rules are covered in the Elastic Security Guide. This page concentrates on the keys your teams create and hand to software, which is where ownership and expiry go wrong. For the general discipline that applies across every platform, see the API Key Management Guide.
Scoping a REST key with role descriptors
The role_descriptors object in the create API key request uses the same structure as the create role API. A key's effective access is the intersection of what the descriptors grant and what the owner holds at the time, so a key can never exceed its owner, but it can equal the owner if you leave the descriptors out.
- cluster: list only the cluster privileges the client needs. Shippers usually need monitoring privileges for their own health checks and nothing that manages security, ILM or snapshots.
- indices: each entry needs
names(index names or patterns) andprivileges. Prefercreate_docorcreatefor append-only ingest overwriteorall, and use a narrow pattern such aslogs-payments-*instead of*. - query and field_security: add a document level security query or a field grant when a reporting client should see only one tenant's documents or a subset of fields.
- allow_restricted_indices: leave at the default
falseso the key cannot touch system indices. - remote_indices: added in 8.14.0 for keys that query remote clusters through the API key based security model; each entry also needs
clusters. - metadata: record the owning team, application, ticket and rotation date. Metadata keys starting with an underscore are reserved for system use. Metadata is queryable, which makes it the cheapest ownership register you can build.
Two behaviours deserve emphasis. First, when a request is itself authenticated with an API key, a derived key must be created with an explicit role descriptor that has no privileges, so an API key cannot mint a more capable child. Second, the grant API key API (POST /_security/api_key/grant, privilege grant_api_key) lets an application create keys for end users by presenting their credentials (password, access token, JWT or a user-managed service account token), optionally with run as. Restrict grant_api_key to the one service that genuinely brokers keys, because it can produce keys for any user whose credentials it holds.
Key names can be up to 256 characters, cannot begin or end with whitespace and cannot begin with an underscore. Adopt a convention such as team-app-env-purpose so a name alone tells a responder whose key it is.
Expiry, rotation and invalidation
Elastic's create, grant and cross-cluster API references each state that keys never expire by default and that expiry is set at creation with expiration in Elasticsearch time units (for example 30d). In Kibana, the advanced creation flow also defaults to no expiry, while the simple creation flow from Help, Connection details on Elastic Cloud Hosted and ECE creates a personal key with 90 days expiry. When a key expires, Kibana shows its status change from Active to Expired.
| Client | Suggested expiry | Rotation pattern |
|---|---|---|
| Interactive or personal key for a script | 7 to 30 days | Recreate on demand; no long-lived personal keys in shared systems |
| Ingest shipper (Beats, Logstash, custom) | 90 days or less | Create new key, deploy, confirm traffic on the new key ID, invalidate old key |
| Application or integration | 90 days or less, aligned with secrets manager rotation | Store in a secrets manager, rotate by overlap, invalidate on cutover |
| Cross-cluster key | Set an expiry agreed by both cluster owners | Create new key on the remote cluster, add to local keystore, reload secure settings, invalidate old key |
These intervals are NHI Mgmt Group suggestions; set yours from your own policy. Rotation is always create, deploy, verify, invalidate, because the update API changes a key's scope, metadata and expiry, not the secret a client presents. The update API also refuses expired or invalidated keys and refuses to be authenticated by an API key, so updates come from the owner's credentials or through run as.
Invalidation uses DELETE /_security/api_key with ids, name, owner=true, or username and realm_name. Privileges decide reach: manage_security can invalidate any key including cross-cluster keys, manage_api_key any REST key, and manage_own_api_key only the caller's own. Invalidated keys fail authentication at once but stay visible in the get and query APIs until the retention period passes; xpack.security.authc.api_key.delete.retention_period defaults to 7d and deletion runs at most every 24h per node.
One operational detail matters during incidents: the API key service caches key IDs and hashes (cache.ttl defaults to 1d, cache.max_keys to 10,000). Invalidation is designed to fail authentication immediately, so test in a non-production cluster that a freshly invalidated key is refused on every node before you rely on it in a runbook. For general rotation design, see the Secret Rotation vs Secret Elimination guide.
Cross-cluster API keys
The API key based remote cluster model (all nodes on 8.14 or later) replaces long-lived trust between clusters with a key that the remote cluster's administrator issues and controls. The remote cluster verifies the key and grants access from the key's own privileges; users on the local cluster can only be further restricted by their local remote_indices privileges, never given more.
- Create the key on the remote cluster: call the cross-cluster API with an
accessobject listing only the index patterns needed undersearchand, if used,replication. Do not add privileges; Elasticsearch convertsaccessinto a role descriptor. The request must use a non-API-key credential andmanage_security. - Open the remote cluster interface deliberately: set
remote_cluster_server.enabled: trueandremote_cluster.hoston remote nodes (off by default on self-managed clusters). The interface listens on port 9443 by default (remote_cluster.port). Allow 9443 only from the local cluster's node addresses. - Encrypt the interface: enable
xpack.security.remote_cluster_server.ssl.enabledon the remote side andxpack.security.remote_cluster_client.ssl.enabledwithcertificate_authoritieson the local side. - Store the key in the keystore, not in YAML: add the encoded key to every local node as
cluster.remote.<alias>.credentials, then call reload secure settings so rotation needs no restart. - Bind the key to a certificate: for strong identity verification, create the key with a
certificate_identitypattern, a regular expression matched against the subject DN of the local cluster's signing certificate (cluster.remote.<alias>.signing.certificateand.key), and trust the signing CA on the remote withcluster.remote.signing.certificate_authorities. The remote then checks chain, DN, signature and expiry. Elastic warns thatCN=.*matches any certificate and is not recommended for production; anchor the pattern to one exact DN.
Because a cross-cluster key cannot authenticate through the REST interface, a copy that leaks is less useful to an attacker than a REST key, but it still exposes every index its access object names to anyone who can reach port 9443. Certificate binding closes most of that gap. Update cross-cluster keys only with the update cross-cluster API key API; the REST key update APIs return an error for them.
Inventory and ownership
The query API key information API (POST /_security/_query/api_key) is the inventory tool. It needs read_security or higher to see every key; with only manage_own_api_key a caller sees their own keys. Queryable fields are id, type, name, creation, expiration, invalidated, invalidation, username, realm and metadata, and aggregations such as terms, missing and date_range run on the same fields.
| Question | Query approach |
|---|---|
| Which active keys never expire? | Filter invalidated: false with a missing aggregation or bool must_not exists on expiration |
| Which keys belong to people who have left? | terms aggregation on username and realm, joined with your leaver list from the identity provider |
| Which keys have no owner metadata? | must_not exists on metadata.owner (or your chosen field) |
| How many cross-cluster keys exist and where? | term on type: cross_cluster |
| What can a key really do? | Request with with_limited_by=true to return the owner's limiting role descriptors alongside the key's own |
Run the inventory on a schedule, export it to your NHI register, and give every key a human owner and a service owner. Paging is capped at 10,000 hits by default, so large estates should use search_after. The ownership process itself is covered in How to assign owners to orphaned service accounts.
Where identity fits
An Elastic API key is a non-human identity with an inherited human shadow: it is owned by a user in a realm, and its maximum reach is that user's role. That creates three identity design choices.
- Who owns automation keys: keys for production shippers should be owned by a dedicated, non-interactive user or created through a broker using the grant API, not by an engineer's SSO account whose roles change with their job.
- API key or service account token: Elastic service accounts and their tokens exist for Elastic's own components; your applications and pipelines should use scoped API keys. See API Keys vs Service Accounts for the wider trade-off.
- Leavers and movers: because the owner's privileges cap the key and an update refreshes the snapshot, removing a person's roles can silently break or reshape the keys they own. Re-home keys before deprovisioning, then invalidate by
usernameandrealm_name.
Detection and audit logging
Elasticsearch audit logging records API key lifecycle changes as security_config_change events, so that event type must be included. Turn on xpack.security.audit.logfile.events.emit_security_config_change_actor, because by default these events do not include actor attribution fields such as user.name, user.realm and authentication.type.
| Event action | Logged when | Payload to alert on |
|---|---|---|
create_apikey | Create or grant API key API creates a key (grant adds a grant object) | create.apikey with empty role descriptors, no expiration, or cluster privileges such as manage_security |
change_apikey | Update API key API changes one key | change.apikey where role descriptors become empty or expiration is removed |
change_apikeys | Bulk update API changes several keys | Large ids lists outside a change window |
invalidate_apikeys | Invalidate API key API runs | Mass invalidation by someone other than the owner (owned_by_authenticated_user: false) |
For use of keys, authentication events carry authentication.type: API_KEY with apikey.id and apikey.name, and user.name identifies the key's owner. Alert on a key ID seen from a new source address or user agent, a shipper key reading indices it normally writes, and access denials for a key that previously worked (a sign of an owner role change). These map to ATT&CK T1078 Valid Accounts and T1552.001 Credentials In Files.
Metrics:
- Percentage of active REST keys with an expiration set (target 100% for new keys).
- Percentage of active keys created with non-empty role descriptors.
- Number of active keys owned by disabled or departed users (target zero).
- Median age of active ingest and application keys against your rotation policy.
- Cross-cluster keys with an exact
certificate_identitypattern, as a share of all cross-cluster keys. - Time from leak detection to confirmed invalidation.
Responding to a leaked Elastic API key
- Identify the key: decode the credential to its key ID (the encoded form is base64 of ID and secret) and look it up with the query API, noting owner, realm, type, role descriptors and
with_limited_byoutput. - Invalidate: call the invalidate API with
ids; confirm the ID appears ininvalidated_api_keys. For a cross-cluster key, a user withmanage_securityon the remote cluster must do this. - Scope the exposure: search audit logs for the
apikey.idacross the window since the key was created or last known safe, listing indices read and any keys it created (derived keys have no privileges, but check). - Replace and redeploy: create a scoped, expiring replacement, store it in a secrets manager and deploy it; remove the old value from repositories, CI variables and images. See the Developer Secret Scanning Triage guide.
- Fix the cause: add Elastic key patterns to secret scanning, and record the lesson against the owning team.
Common failure patterns
- Snapshot keys: keys created with no role descriptors by an administrator, giving a log shipper superuser-equivalent reach.
- Accidental widening: an automation sends
"role_descriptors": {}in an update intending "no change" and the key inherits the owner's full permissions. - No expiry on API-created keys: teams assume the 90 day Kibana default applies everywhere, while keys created through the API or advanced flow have no expiry unless set.
- Personal ownership of production keys: the engineer leaves, their roles are removed, and ingest breaks or the key is never found.
- Wildcard certificate identity: cross-cluster keys bound with
CN=.*, which matches any certificate. - Audit without actors:
security_config_changeevents logged withoutemit_security_config_change_actor, so investigators cannot tell who created a key.
Implementation: a phased rollout
- Inventory (week 1): run the query API with
read_security; list keys with no expiry, no metadata, empty descriptors and departed owners. - Turn on audit attribution: include
security_config_changeand enable actor fields; build the four lifecycle alerts above. - Standardise creation: publish a key template (naming, metadata fields, mandatory
expiration, least privilege descriptors) and restrictmanage_api_keyandgrant_api_keyto a small platform group. - Replace high-risk keys in rings: one non-production environment, then a canary shipper group, then all; watch access denials before invalidating old keys. Roll back by redeploying the previous key, which stays valid until you invalidate it.
- Move cross-cluster links: migrate to API key based remote clusters with TLS and an exact
certificate_identity, and keep keys in the keystore. - Operate: monthly inventory review, quarterly owner attestation, and rotation tracked in your secrets manager.
Small teams can do all of this with the query API and a spreadsheet register. Large estates should create keys only through a broker service using the grant API or infrastructure as code, so every key gets metadata and an expiry by construction.
Framework mapping
| Control | NIST SP 800-53 Rev 5 | NIST CSF 2.0 | ATT&CK |
|---|---|---|---|
| Owner, inventory and leaver handling for keys | AC-2 Account Management | PR.AA-01 | T1078 Valid Accounts |
| Role descriptors and least privilege | AC-6 Least Privilege | PR.AA-05 | T1078 Valid Accounts |
| Expiry, rotation, keystore storage | IA-5 Authenticator Management | PR.AA-01 | T1552.001 Credentials In Files |
| Audit of key lifecycle and use | AU-2 Event Logging, SI-4 System Monitoring | DE.CM-01 | T1078 Valid Accounts |
| TLS and port restriction for cross-cluster traffic | SC-8, SC-7 Boundary Protection | PR.IR-01 |
Practitioner checklist
- Every new REST key has
role_descriptors, anexpirationand owner metadata. - No automation sends an empty
role_descriptorsobject on update. manage_api_key,manage_securityandgrant_api_keyare held by a small, reviewed group.- Production keys are owned by non-interactive users or created through a broker, not personal accounts.
- The query API inventory runs monthly and keys of departed owners are re-homed or invalidated.
- Cross-cluster keys use TLS on port 9443, keystore storage and an exact
certificate_identity. - Audit logging includes
security_config_changewith actor attribution turned on. - A leak runbook covering lookup, invalidation, audit search and replacement has been tested.
Standards and references
- Elastic: Elasticsearch API keys
- Elastic: Create an API key API
- Elastic: Update an API key API
- Elastic: Invalidate API keys API
- Elastic: Create a cross-cluster API key API
- Elastic: Add remote clusters using API key authentication
- Elastic: Find API keys with a query
- Elastic: Elasticsearch audit events
Related NHI Mgmt Group resources: API Key Management Guide, Elastic Security Guide, NHI lifecycle automation.