Join our Newsletter — 33% off our NHI Course
Home› Guides› Elastic API Keys Security Guide: Scoping, Expiry and…
Guide Non-Human Identity (NHI)

Elastic API Keys Security Guide: Scoping, Expiry and Cross Cluster Keys

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 14 November 2026 16 min read
On this page

Elasticsearch API keys are bearer credentials that carry a snapshot of their owner's privileges, and unless you set an expiry when you create them through the API they stay valid until someone invalidates them. They end up in Beats and Elastic Agent configs, Logstash pipelines, application settings, CI variables and remote cluster keystores, often created by a person who later changes role or leaves. This guide covers the lifecycle of Elastic API keys: the three key types, how role descriptors scope a key, expiry and rotation, cross-cluster API keys and their strong identity verification option, inventory with the query API, audit events, and leak response. It is written for platform, search and security engineers who administer Elasticsearch, Kibana or Elastic Cloud deployments, and it goes deeper on keys than the Elastic Security Guide.

This is an independent guide from NHI Mgmt Group, based on Elastic's public documentation as of November 2026. It is not sponsored or reviewed by Elastic. Product features change, so confirm settings against Elastic's current documentation.

Key takeaways

  • Always pass role descriptors. A REST API key created without role_descriptors receives a point-in-time snapshot of everything its owner can do; with descriptors, its access is the intersection of the descriptors and the owner's privileges.
  • Set an expiry every time. Keys created through the create, grant and cross-cluster APIs have no expiry unless you set expiration; the Kibana simple creation flow on Elastic Cloud Hosted and ECE applies 90 days.
  • Treat cross-cluster keys as a separate class. They are created only with manage_security, grant exactly what the access object lists, cannot authenticate REST calls, and can be bound to a signing certificate with certificate_identity.
  • Owner changes reach the key. Updating a key refreshes the owner's privilege snapshot, and sending an empty role_descriptors object removes the restriction so the key inherits the owner's full access.
  • Inventory with the query API, alert on the audit trail. The query API key information API lists every key with owner, realm, creation and expiry, and the create_apikey, change_apikey and invalidate_apikeys audit events show who changed what once actor attribution is turned on.

The three Elastic API key types

Elastic documents three kinds of API key. They share the API key service (enabled by default through xpack.security.authc.api_key.enabled) but differ in who creates them, what they can do and which privilege manages them.

TypePurposeCreated withPrivilege to create or updateHow access is set
User (REST) API keyExternal services call Elasticsearch and Kibana APIs on behalf of a userPOST /_security/api_key, grant API key API, Kibana API keys pagemanage_own_api_key (own keys) or manage_api_keyIntersection of role_descriptors and the owner's privileges, or the owner's full snapshot if none are given
Cross-cluster API keyAnother cluster connects for cross-cluster search or replicationPOST /_security/cross_cluster/api_key, Kibana API keys pagemanage_security; Kibana states an Enterprise licence is neededExactly the access.search and access.replication entries; the creator's privileges are not captured
Managed API keyKibana runs background tasks such as rules and alertingCreated and managed by KibanaManaged by KibanaSnapshot of the user who created or last edited the object

The managed keys behind Kibana rules are covered in the Elastic Security Guide. This page concentrates on the keys your teams create and hand to software, which is where ownership and expiry go wrong. For the general discipline that applies across every platform, see the API Key Management Guide.

Scoping a REST key with role descriptors

The role_descriptors object in the create API key request uses the same structure as the create role API. A key's effective access is the intersection of what the descriptors grant and what the owner holds at the time, so a key can never exceed its owner, but it can equal the owner if you leave the descriptors out.

  • cluster: list only the cluster privileges the client needs. Shippers usually need monitoring privileges for their own health checks and nothing that manages security, ILM or snapshots.
  • indices: each entry needs names (index names or patterns) and privileges. Prefer create_doc or create for append-only ingest over write or all, and use a narrow pattern such as logs-payments-* instead of *.
  • query and field_security: add a document level security query or a field grant when a reporting client should see only one tenant's documents or a subset of fields.
  • allow_restricted_indices: leave at the default false so the key cannot touch system indices.
  • remote_indices: added in 8.14.0 for keys that query remote clusters through the API key based security model; each entry also needs clusters.
  • metadata: record the owning team, application, ticket and rotation date. Metadata keys starting with an underscore are reserved for system use. Metadata is queryable, which makes it the cheapest ownership register you can build.

Two behaviours deserve emphasis. First, when a request is itself authenticated with an API key, a derived key must be created with an explicit role descriptor that has no privileges, so an API key cannot mint a more capable child. Second, the grant API key API (POST /_security/api_key/grant, privilege grant_api_key) lets an application create keys for end users by presenting their credentials (password, access token, JWT or a user-managed service account token), optionally with run as. Restrict grant_api_key to the one service that genuinely brokers keys, because it can produce keys for any user whose credentials it holds.

Key names can be up to 256 characters, cannot begin or end with whitespace and cannot begin with an underscore. Adopt a convention such as team-app-env-purpose so a name alone tells a responder whose key it is.

Expiry, rotation and invalidation

Elastic's create, grant and cross-cluster API references each state that keys never expire by default and that expiry is set at creation with expiration in Elasticsearch time units (for example 30d). In Kibana, the advanced creation flow also defaults to no expiry, while the simple creation flow from Help, Connection details on Elastic Cloud Hosted and ECE creates a personal key with 90 days expiry. When a key expires, Kibana shows its status change from Active to Expired.

ClientSuggested expiryRotation pattern
Interactive or personal key for a script7 to 30 daysRecreate on demand; no long-lived personal keys in shared systems
Ingest shipper (Beats, Logstash, custom)90 days or lessCreate new key, deploy, confirm traffic on the new key ID, invalidate old key
Application or integration90 days or less, aligned with secrets manager rotationStore in a secrets manager, rotate by overlap, invalidate on cutover
Cross-cluster keySet an expiry agreed by both cluster ownersCreate new key on the remote cluster, add to local keystore, reload secure settings, invalidate old key

These intervals are NHI Mgmt Group suggestions; set yours from your own policy. Rotation is always create, deploy, verify, invalidate, because the update API changes a key's scope, metadata and expiry, not the secret a client presents. The update API also refuses expired or invalidated keys and refuses to be authenticated by an API key, so updates come from the owner's credentials or through run as.

Invalidation uses DELETE /_security/api_key with ids, name, owner=true, or username and realm_name. Privileges decide reach: manage_security can invalidate any key including cross-cluster keys, manage_api_key any REST key, and manage_own_api_key only the caller's own. Invalidated keys fail authentication at once but stay visible in the get and query APIs until the retention period passes; xpack.security.authc.api_key.delete.retention_period defaults to 7d and deletion runs at most every 24h per node.

One operational detail matters during incidents: the API key service caches key IDs and hashes (cache.ttl defaults to 1d, cache.max_keys to 10,000). Invalidation is designed to fail authentication immediately, so test in a non-production cluster that a freshly invalidated key is refused on every node before you rely on it in a runbook. For general rotation design, see the Secret Rotation vs Secret Elimination guide.

Cross-cluster API keys

The API key based remote cluster model (all nodes on 8.14 or later) replaces long-lived trust between clusters with a key that the remote cluster's administrator issues and controls. The remote cluster verifies the key and grants access from the key's own privileges; users on the local cluster can only be further restricted by their local remote_indices privileges, never given more.

  1. Create the key on the remote cluster: call the cross-cluster API with an access object listing only the index patterns needed under search and, if used, replication. Do not add privileges; Elasticsearch converts access into a role descriptor. The request must use a non-API-key credential and manage_security.
  2. Open the remote cluster interface deliberately: set remote_cluster_server.enabled: true and remote_cluster.host on remote nodes (off by default on self-managed clusters). The interface listens on port 9443 by default (remote_cluster.port). Allow 9443 only from the local cluster's node addresses.
  3. Encrypt the interface: enable xpack.security.remote_cluster_server.ssl.enabled on the remote side and xpack.security.remote_cluster_client.ssl.enabled with certificate_authorities on the local side.
  4. Store the key in the keystore, not in YAML: add the encoded key to every local node as cluster.remote.<alias>.credentials, then call reload secure settings so rotation needs no restart.
  5. Bind the key to a certificate: for strong identity verification, create the key with a certificate_identity pattern, a regular expression matched against the subject DN of the local cluster's signing certificate (cluster.remote.<alias>.signing.certificate and .key), and trust the signing CA on the remote with cluster.remote.signing.certificate_authorities. The remote then checks chain, DN, signature and expiry. Elastic warns that CN=.* matches any certificate and is not recommended for production; anchor the pattern to one exact DN.

Because a cross-cluster key cannot authenticate through the REST interface, a copy that leaks is less useful to an attacker than a REST key, but it still exposes every index its access object names to anyone who can reach port 9443. Certificate binding closes most of that gap. Update cross-cluster keys only with the update cross-cluster API key API; the REST key update APIs return an error for them.

Inventory and ownership

The query API key information API (POST /_security/_query/api_key) is the inventory tool. It needs read_security or higher to see every key; with only manage_own_api_key a caller sees their own keys. Queryable fields are id, type, name, creation, expiration, invalidated, invalidation, username, realm and metadata, and aggregations such as terms, missing and date_range run on the same fields.

QuestionQuery approach
Which active keys never expire?Filter invalidated: false with a missing aggregation or bool must_not exists on expiration
Which keys belong to people who have left?terms aggregation on username and realm, joined with your leaver list from the identity provider
Which keys have no owner metadata?must_not exists on metadata.owner (or your chosen field)
How many cross-cluster keys exist and where?term on type: cross_cluster
What can a key really do?Request with with_limited_by=true to return the owner's limiting role descriptors alongside the key's own

Run the inventory on a schedule, export it to your NHI register, and give every key a human owner and a service owner. Paging is capped at 10,000 hits by default, so large estates should use search_after. The ownership process itself is covered in How to assign owners to orphaned service accounts.

Where identity fits

An Elastic API key is a non-human identity with an inherited human shadow: it is owned by a user in a realm, and its maximum reach is that user's role. That creates three identity design choices.

  • Who owns automation keys: keys for production shippers should be owned by a dedicated, non-interactive user or created through a broker using the grant API, not by an engineer's SSO account whose roles change with their job.
  • API key or service account token: Elastic service accounts and their tokens exist for Elastic's own components; your applications and pipelines should use scoped API keys. See API Keys vs Service Accounts for the wider trade-off.
  • Leavers and movers: because the owner's privileges cap the key and an update refreshes the snapshot, removing a person's roles can silently break or reshape the keys they own. Re-home keys before deprovisioning, then invalidate by username and realm_name.

Detection and audit logging

Elasticsearch audit logging records API key lifecycle changes as security_config_change events, so that event type must be included. Turn on xpack.security.audit.logfile.events.emit_security_config_change_actor, because by default these events do not include actor attribution fields such as user.name, user.realm and authentication.type.

Event actionLogged whenPayload to alert on
create_apikeyCreate or grant API key API creates a key (grant adds a grant object)create.apikey with empty role descriptors, no expiration, or cluster privileges such as manage_security
change_apikeyUpdate API key API changes one keychange.apikey where role descriptors become empty or expiration is removed
change_apikeysBulk update API changes several keysLarge ids lists outside a change window
invalidate_apikeysInvalidate API key API runsMass invalidation by someone other than the owner (owned_by_authenticated_user: false)

For use of keys, authentication events carry authentication.type: API_KEY with apikey.id and apikey.name, and user.name identifies the key's owner. Alert on a key ID seen from a new source address or user agent, a shipper key reading indices it normally writes, and access denials for a key that previously worked (a sign of an owner role change). These map to ATT&CK T1078 Valid Accounts and T1552.001 Credentials In Files.

Metrics:

  • Percentage of active REST keys with an expiration set (target 100% for new keys).
  • Percentage of active keys created with non-empty role descriptors.
  • Number of active keys owned by disabled or departed users (target zero).
  • Median age of active ingest and application keys against your rotation policy.
  • Cross-cluster keys with an exact certificate_identity pattern, as a share of all cross-cluster keys.
  • Time from leak detection to confirmed invalidation.

Responding to a leaked Elastic API key

  1. Identify the key: decode the credential to its key ID (the encoded form is base64 of ID and secret) and look it up with the query API, noting owner, realm, type, role descriptors and with_limited_by output.
  2. Invalidate: call the invalidate API with ids; confirm the ID appears in invalidated_api_keys. For a cross-cluster key, a user with manage_security on the remote cluster must do this.
  3. Scope the exposure: search audit logs for the apikey.id across the window since the key was created or last known safe, listing indices read and any keys it created (derived keys have no privileges, but check).
  4. Replace and redeploy: create a scoped, expiring replacement, store it in a secrets manager and deploy it; remove the old value from repositories, CI variables and images. See the Developer Secret Scanning Triage guide.
  5. Fix the cause: add Elastic key patterns to secret scanning, and record the lesson against the owning team.

Common failure patterns

  • Snapshot keys: keys created with no role descriptors by an administrator, giving a log shipper superuser-equivalent reach.
  • Accidental widening: an automation sends "role_descriptors": {} in an update intending "no change" and the key inherits the owner's full permissions.
  • No expiry on API-created keys: teams assume the 90 day Kibana default applies everywhere, while keys created through the API or advanced flow have no expiry unless set.
  • Personal ownership of production keys: the engineer leaves, their roles are removed, and ingest breaks or the key is never found.
  • Wildcard certificate identity: cross-cluster keys bound with CN=.*, which matches any certificate.
  • Audit without actors: security_config_change events logged without emit_security_config_change_actor, so investigators cannot tell who created a key.

Implementation: a phased rollout

  1. Inventory (week 1): run the query API with read_security; list keys with no expiry, no metadata, empty descriptors and departed owners.
  2. Turn on audit attribution: include security_config_change and enable actor fields; build the four lifecycle alerts above.
  3. Standardise creation: publish a key template (naming, metadata fields, mandatory expiration, least privilege descriptors) and restrict manage_api_key and grant_api_key to a small platform group.
  4. Replace high-risk keys in rings: one non-production environment, then a canary shipper group, then all; watch access denials before invalidating old keys. Roll back by redeploying the previous key, which stays valid until you invalidate it.
  5. Move cross-cluster links: migrate to API key based remote clusters with TLS and an exact certificate_identity, and keep keys in the keystore.
  6. Operate: monthly inventory review, quarterly owner attestation, and rotation tracked in your secrets manager.

Small teams can do all of this with the query API and a spreadsheet register. Large estates should create keys only through a broker service using the grant API or infrastructure as code, so every key gets metadata and an expiry by construction.

Framework mapping

ControlNIST SP 800-53 Rev 5NIST CSF 2.0ATT&CK
Owner, inventory and leaver handling for keysAC-2 Account ManagementPR.AA-01T1078 Valid Accounts
Role descriptors and least privilegeAC-6 Least PrivilegePR.AA-05T1078 Valid Accounts
Expiry, rotation, keystore storageIA-5 Authenticator ManagementPR.AA-01T1552.001 Credentials In Files
Audit of key lifecycle and useAU-2 Event Logging, SI-4 System MonitoringDE.CM-01T1078 Valid Accounts
TLS and port restriction for cross-cluster trafficSC-8, SC-7 Boundary ProtectionPR.IR-01

Practitioner checklist

  • Every new REST key has role_descriptors, an expiration and owner metadata.
  • No automation sends an empty role_descriptors object on update.
  • manage_api_key, manage_security and grant_api_key are held by a small, reviewed group.
  • Production keys are owned by non-interactive users or created through a broker, not personal accounts.
  • The query API inventory runs monthly and keys of departed owners are re-homed or invalidated.
  • Cross-cluster keys use TLS on port 9443, keystore storage and an exact certificate_identity.
  • Audit logging includes security_config_change with actor attribution turned on.
  • A leak runbook covering lookup, invalidation, audit search and replacement has been tested.

Standards and references

Related NHI Mgmt Group resources: API Key Management Guide, Elastic Security Guide, NHI lifecycle automation.

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 14 November 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.