Datadog API and Application Keys Security Guide: Scoping, Rotation and Leak Response

Datadog API keys and application keys are long-lived machine credentials with no built-in expiry, and they are routinely copied into Agent configs, CI variables, Terraform state and scripts where nobody tracks them. An API key lets anyone submit telemetry into your organisation, and an application key paired with it can read dashboards, logs, monitors and configuration with the full permissions of the person who created it. This guide explains how the Datadog credential types differ, which RBAC permissions control them, how to scope and rotate them, what the Audit Trail records, and how to respond when a key leaks. It is written for platform, SRE and security engineers who administer a Datadog organisation, and it goes deeper than the general SaaS API Token Security Guide.

Key takeaways

• Two keys, two risks. API keys are organisation-level and used by the Agent to submit data; application keys belong to a user and, unless scoped, carry all of that user's permissions.
• Scope every application key. An unscoped key inherits all of its creator's scopes and permissions, and scopes do not shrink later if the user's role changes.
• Move automation to service accounts. Personal application keys are revoked when the user is disabled; service account keys and Service Access Tokens survive staff changes and can be governed centrally.
• Nothing expires unless you make it. API and application keys have no built-in expiry, so rotation is a process you own, ideally driven from a secrets manager.
• Know the revocation asymmetry. A revoked API key can be restored for 7 days; a revoked application key is gone permanently. Plan your leak response around that difference.

How identity and access work in Datadog

Human users sign in through the web UI with a password, Google login or SAML, and receive permissions through roles (Datadog Admin, Datadog Standard, Datadog Read Only, or custom roles). Machine access uses a separate set of credentials, each with a different owner, purpose and lifecycle.

• API keys are not secret-free: they cannot read your data on their own, but they can submit fake metrics, logs and events, poison dashboards and monitors, and drive up ingestion costs. Datadog states that API keys must never be shipped in browser, mobile or TV apps; that is what client tokens are for.
• Application keys act as a person: by default a key has every permission of its owner. A key created by an administrator is, in effect, an administrator credential that never expires.
• Limits: an organisation must keep at least one API key and has a default limit of 50 (Managed keys do not count). Key names must be unique across the organisation.
• Eventual consistency: creation and revocation take a few seconds to propagate, so a brand-new key may return 401 or 403 briefly. Datadog recommends retries with short exponential backoff, and provides /api/v1/validate to test an API key and /api/v2/validate_keys to test an application key.

Admin roles and privileged access

Key management is controlled by specific RBAC permissions. Because Datadog Admin includes all of them, the number of Admin role holders is the first thing to reduce.

• Custom roles: build a "Standard minus keys" custom role that removes api_keys_read and user_app_keys for engineers who only need the UI, and keep key creation with a small platform team.
• Delegation guard: when creating a service account you can only grant roles and permissions you hold yourself, unless you have user_access_manage. That stops casual privilege escalation through service accounts, but anyone with User Access Manage can bypass it.
• SAML strict: disabling the other login methods makes the organisation SAML strict, so password and Google logins stop working. Use per-user overrides only for break-glass accounts. Check the default role for just-in-time provisioned users: it is Standard unless you change it, which means new users can read API key values. See the Privileged Access Management Guide.

Integrations, API tokens and service accounts

Scoping application keys

Scopes restrict an application key to named authorisation scopes. If you leave scopes unspecified, the key receives all of its creator's scopes. Scope names are case-sensitive, a call outside the scopes returns 403 Forbidden, and an owner cannot grant a scope they do not hold. One subtle point: if the owner's role changes later, scopes already on the key do not change, so review keys when people move teams. Scope keys on creation, because tightening the scopes of a live key can break the integration that uses it.

Service accounts and Service Access Tokens

• Why service accounts: disabling a user revokes the application keys they created, which silently breaks any automation built on a personal key. Datadog does not transfer application keys between users; its answer is a service account, which owns keys independently of staff changes.
• Service Access Tokens: SATs belong to a service account, are always scoped, can carry an expiry, and are sent as Authorization: Bearer with no API key. Up to 500 can exist per service account. A SAT cannot create or update application keys, and it can only create another SAT with a subset of its own scopes and an expiry no later than its own, which limits how far a stolen SAT can extend itself. Datadog's service account documentation describes SATs as the recommended method for API authentication.
• Notifications: Datadog emails the service account address when a key is created or revoked or the account is disabled. Point that address at a monitored team mailbox, not an individual.

One-Time Read mode and storage

One-Time Read (OTR) mode shows an application key secret only once, at creation. Parent organisations created after 20 August 2025 have it on permanently. Existing organisations can enable it under Organization Settings, Application Keys (requires org_app_keys_write and org_management); it can be switched off for 3 months, after which it becomes permanent. On the ap2 and ddog-gov sites secrets are already unretrievable. OTR removes the habit of reading keys back from the UI, so every key must go into a secrets manager at creation. Datadog names AWS Secrets Manager, HashiCorp Vault and Azure Key Vault, and notes that AWS Secrets Manager supports managed rotation for Datadog keys. See the Secrets Management Guide.

Choosing a credential

Small organisations can start with one service account per pipeline and a quarterly rotation; larger ones with child organisations should treat each child as a separate trust boundary, because settings such as the IP allowlist do not inherit between parent and child. See the Service Account Security Guide.

Logging and detection

Audit Trail is the record of who created, viewed, used and revoked keys. It is enabled under Organization Settings, Audit Trail Settings by a user with audit_logs_write. Default retention when enabled is 90 days (options are 3, 7, 15, 30 or 90 days), and you can archive events to Amazon S3, Google Cloud Storage or Azure Storage for longer retention and SIEM ingestion. Request events record API calls, and calls made with a SAT are attributed to the owning service account.

• Monitors on audit events: create Audit Trail monitors for the queries above and route them to the security on-call, not the observability team that holds the keys.
• Last used: the Last used timestamp on application keys is visible to all customers; Audit Trail extends how far back usage can be traced. Use it to find stale keys.
• Ingest anomalies: a leaked API key often shows up as unexpected hosts, services or log sources, or an unexplained jump in usage, rather than in Audit Trail. Watch for new host names and sources that do not match your inventory.
• ATT&CK view: exposure of keys in repositories and config files maps to T1552.001 Credentials In Files and T1528 Steal Application Access Token; an attacker adding keys or SATs to keep access maps to T1098.001 Additional Cloud Credentials. See the Cloud Logging and Audit Trail Guide.

Metrics that show the programme is working:

1. Percentage of application keys and SATs owned by service accounts rather than people.
2. Percentage of application keys with explicit scopes (target 100%).
3. Number of keys not used in 90 days, trending to zero.
4. Number of users holding api_keys_read or org_app_keys_write.
5. Median age of API keys against your rotation standard.
6. Time from leak detection to revocation and replacement.

Hardening baseline

1. Inventory first: export every API key, application key, SAT and client token with owner, last used, scopes and where it is deployed. Tag each to a system in your Service Account Inventory Template.
2. Enable Audit Trail and archive it before changing anything, so rotation work is recorded and you have a baseline of normal key usage.
3. Turn on One-Time Read mode after confirming every consumer can read keys from a secret store. Use the 3-month reversible window as your pilot period.
4. Split API keys so each deployment method or environment has its own key, which lets you revoke one without taking down every Agent.
5. Migrate automation from personal application keys to service accounts with scoped SATs or scoped keys. Pilot on one pipeline, run old and new credentials in parallel, then revoke the personal key.
6. Remove default exposure: create custom roles without api_keys_read and user_app_keys for users who do not need them, and reduce Datadog Admin holders.
7. Enforce SAML strict with documented break-glass overrides, and consider the IP allowlist (Pro+ and Enterprise, enabled through Datadog support). Remember it covers the UI, public API, OAuth integrations and the Datadog MCP Server but not Agent ingest endpoints or the API key validate endpoint, so it does not stop a leaked API key from submitting data.
8. Set a rotation standard: for example 90 days for application keys and 180 days for API keys, using managed rotation where available. Rollback is simple for API keys within the 7-day Unrevoke window; for application keys, keep the old key live until the new one is confirmed working, because revocation cannot be undone.

Common misconfigurations

• Admin personal keys in CI: an administrator's unscoped application key used by Terraform is an admin credential with no expiry. When that person leaves and is disabled, the key is revoked and pipelines fail, which often leads teams to keep leavers' accounts active instead of fixing the dependency.
• Assuming disabling a user removes everything: disabling a user revokes their application keys, but API keys they created remain valid. Offboarding must include a review of API keys by creator. See the OWASP NHI1 Improper Offboarding Guide.
• Keys in Helm values, Dockerfiles and repositories: Datadog's own guidance is to keep keys out of source code and environment files. A public repository with a key is a leak even if the file is deleted, as Datadog warns that removing a file does not guarantee nobody copied the key. See the OWASP NHI2 Secret Leakage Guide.
• Partial rotation after an incident: rotating the obvious key and leaving others found in the same source is a recurring failure. In the Internet Archive Breach, API tokens exposed with a GitLab token were reportedly still unrotated weeks later and were used for a second intrusion.
• API key in a mobile or browser app: any value shipped to a client is public. Use client tokens, which are designed for that exposure.
• One API key everywhere: a single key across production, staging and every cloud means a leak forces a fleet-wide emergency change. Long-lived shared keys are the pattern described in the OWASP NHI7 Long-Lived Secrets Guide.

Leak response

Datadog's documented sequence, with practical additions:

1. Revoke the exposed key. If usage is large or unclear, complete steps 2 to 5 first so revocation does not cause an outage, but keep that window short.
2. Remove the key from public locations, publish a sanitised version and purge it from commit history.
3. Create a new key, scoped and owned by a service account where possible, stored in a secrets manager.
4. Update affected services and confirm with the validate endpoints and Agent status.
5. Review the organisation for unapproved access: recently added users, new resources, role or permission changes, new keys or SATs, and Audit Trail request events from unfamiliar IP addresses.
6. Look wider: check the same file, repository and machine for other secrets and rotate them in the same change window. See the Secrets Rotation Runbook Template.

Framework mapping

In CIS Controls v8.1 this work sits under Control 5 Account Management and Control 6 Access Control Management.

Practitioner checklist

• Enable Audit Trail, set 90-day retention and archive events to cloud storage feeding your SIEM.
• Inventory every API key, application key, SAT and client token with owner, scopes, last used and deployment location.
• Replace personal application keys used by automation with service account SATs or scoped keys.
• Apply explicit scopes to every application key; reject unscoped keys in review.
• Give each environment or deployment method its own API key, injected from a secrets manager.
• Enable One-Time Read mode once consumers no longer read keys from the UI.
• Remove api_keys_read and user_app_keys from roles that do not need them and cut Datadog Admin holders.
• Enforce SAML strict with documented break-glass overrides, and evaluate the IP allowlist.
• Create Audit Trail monitors for key creation, role changes, login method changes and Audit Trail setting changes.
• Rehearse the leak response once a year, including the 7-day API key Unrevoke option and the permanent application key revocation.

Standards and references

• Datadog: API and Application Keys
• Datadog: Service Access Tokens
• Datadog: Service Accounts
• Datadog: Datadog Role Permissions
• Datadog: Audit Trail Events
• Datadog: IP Allowlist
• Datadog: Single Sign On With SAML
• NIST: SP 800-53 Rev. 5 Security and Privacy Controls

Related NHI Mgmt Group resources: SaaS API Token Security Guide · API Key Management Guide · Service Account Security Guide · Atlassian API Tokens Security Guide