PagerDuty credentials: How to Revoke Leaked Keys and Prevent Alert Tampering

A leaked PagerDuty credential lets an attacker read who is on call and how your services are wired, and, with write access, quietly change the alerting that would have told you about the rest of their intrusion. PagerDuty sits at the end of most detection pipelines, so its keys are rarely treated as sensitive, yet they turn up in monitoring configs, Terraform state, CI variables and chat bots. This guide covers the four PagerDuty credential families (account REST API keys, user API tokens, OAuth app credentials and Events API integration keys), what each one reaches, how to tell them apart when one is found, the signals to look for, and the exact revoke and rotate steps from PagerDuty's documentation. It is written for SRE, platform and security teams who own a PagerDuty account. For the generic process, start with the Leaked Credential Response Playbook; this page adds the PagerDuty specifics.

Key takeaways

• Account keys are not tied to a person. A General Access REST API key belongs to the account, survives its creator leaving and, unless created read-only, can create, update and delete almost anything.
• Integration keys are a different risk. A 32-character Events API integration key cannot call the REST API, but it can trigger, acknowledge and resolve incidents on its service, which is enough to page people at 3am or close a real alert.
• Disable first, then delete. General Access keys can be disabled reversibly from the API Access Keys table, which gives you a safe containment step before the irreversible Remove.
• Deleting an OAuth client does not revoke its tokens. For OAuth apps, use Revoke All Tokens before you delete and recreate the OAuth functionality to replace a client secret.
• The audit trail has gaps that matter here. PagerDuty's Audit Trail Report does not capture maintenance windows, disabled services or added integrations, which are exactly the changes an attacker would make to silence alerting.

How identity and access work in PagerDuty

PagerDuty's REST API accepts a token in the Authorization header in the exact form Token token=<value>, over HTTPS only; a malformed header returns 401 Unauthorized. The Events API v2 is separate: it takes an integration key in the event body and is meant for monitoring tools to send machine events. PagerDuty's documentation describes these credential families:

Token lifetimes differ sharply. REST API keys have no expiry described in PagerDuty's documentation and live until disabled or removed. Classic User OAuth access tokens last 30 days with 210-day refresh tokens and a three-year rolling window. Scoped OAuth access tokens last one day, user refresh tokens 30 days with a one-year rolling window, and app tokens are simply re-requested with the client credentials when they expire. That makes the Scoped OAuth client_secret, not the token, the long-lived secret to protect. See the API Key Management Guide for the lifecycle model behind these choices.

Admin roles and privileged access

The roles that matter for credentials are the ones that can mint or destroy them.

• Admin user tokens are hard to remove: Admins and Account Owners cannot delete user tokens belonging to the Account Owner, an Admin or a Global Admin. If an admin's token leaks, that admin (or the owner, for others) must remove it, so build that into on-call for the incident.
• Keep the admin set small: every Admin can create account-wide keys that outlive them. Review membership of Owner, Global Admin and Admin quarterly, as the Privileged Access Management Guide recommends for any privileged tier.

Integrations, API tokens and service accounts

Choosing a credential

Scoped OAuth app tokens are requested with grant_type=client_credentials and a scope list that must include as_account-us.<subdomain> or the eu equivalent. PagerDuty notes that an invalid scope does not raise an error: the token silently gets only openid, and calls then fail with 403. That behaviour is useful in an investigation too, because a 403 burst from an app is often a misconfigured script rather than an attacker. Scoped apps must be confidential clients and use PKCE for user tokens, and the client secret must never sit in client-side code.

Where PagerDuty secrets leak

• Monitoring configuration: integration keys in alert manager, uptime checker and logging tool configs, often committed with the rest of the config repository.
• Infrastructure as code: provider tokens in Terraform variables and state, and in CI variables printed by debug logging.
• Chat and runbook bots: user tokens embedded in ChatOps bots that someone set up years ago under their own account.
• Webhook destinations: custom headers on webhook subscriptions are redacted when read back through the API but are sent in clear inside each delivered webhook, so a secret in a custom header is exposed to every system on the delivery path.

The Terraform and IaC Pipeline Credential Security Guide covers the state file problem in depth.

Logging and detection

Recognising a PagerDuty secret

PagerDuty REST keys are short and have no fixed public prefix, so most generic scanners only catch them through context (a variable named PAGERDUTY_TOKEN, a Token token= header). GitHub secret scanning lists two PagerDuty types, both for private repositories with GitHub Advanced Security and both with push protection:

Because PagerDuty is not listed as a partner for public repositories, a key pushed to a public repository is not automatically reported to PagerDuty: nobody revokes it for you. The open source Gitleaks default rule set has no PagerDuty rule, so add a custom rule keyed on your variable names and the Token token= header. See the Git Secret Scanning Guide and the Developer Secret Scanning Triage for rule tuning and alert handling.

Misuse signals in PagerDuty

• Audit Trail Report: on Business, Enterprise for Incident Management and legacy Digital Operations plans, it records create, update and delete actions on Users, Teams, Escalation Policies, Schedules, Services and Incident Workflows, with the actor and a Method field of Browser, API or OAuth (mobile). Changes made with a leaked key appear with Method API. Records cover the last 12 months, viewable 31 days at a time, and are also available through the REST API's audit records endpoint.
• User token last used: the User Settings API Access table shows a last used date for each user token. A last used date after the suspected exposure, on a token whose owner was not working, is a strong signal.
• Incident timeline: unexpected incidents triggered, or real incidents acknowledged or resolved, through one integration point to a leaked integration key.
• What the audit trail misses: PagerDuty lists as not captured: escalation policy user and level changes, schedule overrides, adding or removing service integrations and extensions, disabling services and maintenance windows. Check these by hand in an incident: list maintenance windows, disabled services and recently added integrations, and compare escalation policies with your configuration as code.

Metrics

• Number of full-access General Access keys, trending down as automation moves to scoped OAuth or read-only keys.
• Percentage of keys with a named owner and documented purpose in your inventory.
• User tokens not used in 90 days (from the last used column), and their removal rate.
• Mean time from a PagerDuty secret alert to disable or removal.
• Escalation policy and service drift found between PagerDuty and configuration as code.

Leak response: revoke and rotate

1. Classify the secret: a 32-character value next to an Events API URL or routing_key is an integration key; a 20-character value used as Token token= is a REST key (account or user); a client_secret or bearer token from identity.pagerduty.com is OAuth. Look the key up in the API Access Keys table by description and creation date.
2. Contain an account key: an Admin or Account Owner disables it from Integrations, Developer Tools, API Access Keys. Disabling blocks REST use immediately and is reversible, and the table records who disabled it and when. Create a replacement (shown only once, so put it straight into your vault), redeploy, confirm, then Remove the old key.
3. Contain a user token: remove it under User Settings, API Access, Remove. If the token belongs to an Admin, Global Admin or the Account Owner, that person must remove it themselves. Treat the user's own session as possibly exposed too if the token came from their machine.
4. Contain an OAuth app: in App Registration, open the OAuth client and use Revoke All Tokens. If the client_secret leaked, then delete and recreate the OAuth functionality to get a new secret; deleting alone leaves existing tokens valid.
5. Contain an integration key: on the service's Integrations tab, add a new integration, point the monitoring tool at the new key, verify an end-to-end test event, then delete the old integration. Expect a short window where both exist.
6. Hunt for tampering: filter the Audit Trail Report for Method API from the exposure date, then check the uncaptured areas by hand: maintenance windows, disabled services, schedule overrides, new integrations and extensions, and escalation policy membership. Any of these can silence alerting for other parts of an attack.
7. Check data exposure: user and on-call data (names, emails, phone numbers) is useful for targeted phishing and SIM-swap attempts against your responders. Warn on-call staff if a full-access or user token leaked.
8. Clean the source and record it: remove the secret from code history, images and logs, and record cause and fix. The Secrets Rotation Runbook Template gives the evidence fields.

Hardening baseline

1. Inventory: export the API Access Keys table and every user's API Access list, and the registered OAuth apps. Record owner, purpose and consumer for each.
2. Downgrade: recreate reporting keys as Read-only; move automation that needs writes to Scoped OAuth apps with the minimum scopes, which gives one-day access tokens instead of a non-expiring key.
3. Remove personal tokens from shared automation: bots and scripts running on someone's user token break when that person leaves, and in the meantime act with their full permissions.
4. Vault everything: keys are shown once; store them in your secrets manager and inject at run time. See the Secrets Management Guide.
5. Manage configuration as code: keep services, escalation policies and schedules in code so that drift, including changes the audit trail does not record, shows up in a plan.
6. Test and roll back: disable a candidate key for a few days before removing it. If something breaks, re-enable it, which is the documented rollback path.

Small teams can stop at one read-only key for reporting, integration keys per tool, no personal tokens in automation and secret scanning. Larger organisations should standardise on Scoped OAuth apps for automation, alert on audit records with Method API outside change windows, and run a weekly drift check on alert routing.

Common misconfigurations

• Full-access account key for a dashboard: a read-only need served by a key that can delete services.
• REST key used where an integration key would do: monitoring tools only need to send events.
• Deleting the OAuth client and assuming the tokens died: PagerDuty states that deleting OAuth functionality does not revoke tokens.
• Relying on the audit trail alone: maintenance windows and disabled services, the quickest ways to silence alerts, are not recorded there.
• Secrets in webhook custom headers: redacted in the API, visible in every delivered payload.
• Assuming public repository leaks are auto-revoked: PagerDuty is not a GitHub public partner, so revocation is on you.

Framework mapping

Practitioner checklist

• Export every General Access key, user token and OAuth app, and assign an owner to each.
• Recreate reporting keys as Read-only and move write automation to Scoped OAuth apps.
• Replace user tokens in shared bots and scripts with account-level or OAuth credentials.
• Use Events API integration keys, not REST keys, for monitoring tools.
• Add custom scanner rules for PagerDuty variable names and the Token token= header.
• Make sure at least two responders hold Admin rights to disable keys out of hours.
• Write the OAuth path into your runbook: Revoke All Tokens, then recreate the client secret.
• After any leak, check maintenance windows, disabled services, integrations and escalation policies by hand.
• Review user token last used dates quarterly and remove idle tokens.

Standards and references

• PagerDuty: API Access Keys
• PagerDuty: REST API authentication
• PagerDuty: OAuth Functionality
• PagerDuty: Private Apps (Scoped OAuth)
• PagerDuty: Audit Trail Reporting
• GitHub Docs: Supported secret scanning patterns
• NIST: SP 800-53 Rev 5, Security and Privacy Controls

Related NHI Mgmt Group resources: Leaked Credential Response Playbook · API Key Management Guide · Git Secret Scanning Guide · Secrets Rotation Runbook Template