Third-party apps connected to SaaS tenants through OAuth keep working, and keep their access, long after the person who approved them has forgotten them or left. This health check is a repeatable audit of that estate: ten metrics for OAuth app grants, API keys and SaaS integrations, each with a formula, a data source type and green, amber and red thresholds, plus a cadence, a findings log and a remediation map. It is written for the SaaS security or identity team that runs the review, the internal audit or second-line function that checks it, and the CISO who signs it off. The controls themselves are set out in the SaaS and OAuth App Governance Guide; this page measures whether they are working.
What this health check covers
Scope is every non-human connection into your sanctioned SaaS tenants: OAuth apps and connected apps that users or administrators consented to, marketplace integrations, API keys and personal access tokens issued inside SaaS platforms, and the service accounts those integrations run as. It measures the grants, not the apps' own security. Reviewing a new SaaS application before purchase is a separate job, covered by the SaaS Application Security Review Checklist; this health check takes over once integrations are live and asks whether access has drifted.
If you searched for an OAuth app audit, a third-party app access review or a set of SaaS security KPIs, this is the run sheet. It is deliberately narrow: ten numbers that two auditors would calculate the same way, rather than a full catalogue. Workforce identity measures such as SSO and MFA coverage across all applications sit in the IAM programme health check; only the SaaS integration view of single sign-on is measured here (metric S10).
Key takeaways
- The inventory is the first metric: every other number is only as good as the list of grants it is calculated from, so reconcile the tenant exports before scoring anything.
- Scope beats count: a hundred read-only calendar apps matter less than five apps with mailbox, file or CRM write access, so report high-scope grants separately.
- Grants outlive people: dormant grants and grants made by leavers are the two numbers that most often find access nobody would approve today.
- Tokens and keys are part of the estate: long-lived refresh tokens and SaaS API keys are what attackers reuse after a vendor is breached, so measure their age and ownership too.
- Thresholds are starting points: the bands below are NHI Mgmt Group recommendations; baseline your own tenants and record why you change them.
How to use this template
Copy the metrics table and findings log into your own tracker and run this SaaS and OAuth Integrations Health Check at the cadence below. Collect each figure from the data source types listed, score it against the thresholds, and log every amber or red result as a finding with an owner and a due date. Keep each completed run: the quarter-on-quarter trend shows whether grant hygiene is improving or whether new consents are outpacing clean-up. Replace bracketed placeholders such as [Organisation] and [Tenant] with your own values.
Cadence and sign-off
| Review | Frequency | Owner | Approver | Evidence kept |
|---|---|---|---|---|
| High-scope and unverified app sweep (S2, S3) | Monthly | SaaS security lead | Identity programme owner | Tenant OAuth grant export with scopes, publisher status and consent type |
| Leaver grant check (S5) | Monthly, joined to the HR leaver list | IAM operations | Identity programme owner | HR leaver extract, grant export, revocation records |
| Full metric run (S1 to S10) | Quarterly | SaaS security lead | CISO | Metric workbook, findings log, sign-off record |
| Admin-consented app review (S6) | Quarterly, and within 5 working days of a vendor breach notice | Application owners | SaaS security lead | Review decisions per app, scope justification, revocation evidence |
| Independent re-performance | Yearly | Internal audit or second line | Audit committee | Re-performed samples, variance notes |
Metrics
NHI Mgmt Group recommended thresholds; adjust to your risk appetite. "In-scope tenants" means the sanctioned SaaS platforms listed in your application inventory that support third-party OAuth or API access.
| ID | Metric | Formula (numerator / denominator) | Data source type | Green | Amber | Red |
|---|---|---|---|---|---|---|
| S1 | OAuth app inventory completeness | In-scope tenants whose OAuth grants are exported into the central integration register at least monthly / in-scope tenants | Application inventory; integration register; tenant OAuth grant exports | 100% | 90% to below 100% | Below 90% |
| S2 | High-scope grant share | Active grants holding a scope you classify as high risk (mail read or send, file read or write across the tenant, directory write, CRM export, admin) / active grants | Tenant OAuth grant export with scope list; your scope risk classification | 5% or below, each with a named business owner | Above 5% to 15% | Above 15%, or any high-scope grant with no owner |
| S3 | Unverified publisher apps | Active grants to apps whose publisher is not verified by the SaaS platform / active grants | Tenant OAuth grant export with publisher verification field | 2% or below, none high scope | Above 2% to 10%, none high scope | Above 10%, or any unverified app with a high-risk scope |
| S4 | Dormant grants | Active grants with no token use or API call in the last 90 days / active grants | SaaS audit logs (token issue and API activity by app); grant export | 5% or below | Above 5% to 20% | Above 20% |
| S5 | Grants by leavers | Active user-consented grants whose consenting user left the organisation more than 7 days ago / active user-consented grants | Grant export with consenting user; HR leaver extract | 0% | Above 0% to 1% | Above 1%, or any such grant with a high-risk scope |
| S6 | Admin-consented apps reviewed | Tenant-wide (admin-consented) apps with a recorded owner review in the last 12 months / tenant-wide apps | Grant export filtered to admin consent; review records | 100% | 90% to below 100% | Below 90% |
| S7 | Token age | 95th percentile age in days of active refresh tokens and integration credentials issued to third-party apps, from issue or last rotation | Token or credential metadata from the tenant; integration register | 90 days or less | Above 90 to 365 days | Above 365 days, or tokens with no recorded issue date |
| S8 | API keys per SaaS | Active API keys and personal access tokens in the tenant / active integrations registered for that tenant; reported per tenant, with the share that have an owner and expiry | Tenant API key or token list; integration register | Ratio 1.0 or below, 100% with owner and expiry | Ratio above 1.0 to 2.0, or 90% to below 100% with owner and expiry | Ratio above 2.0, or below 90% with owner and expiry |
| S9 | SSPM misconfiguration count | Open high or critical posture findings that relate to integrations (third-party access settings, consent policy, API access, token lifetime) / in-scope tenants | SaaS posture management findings or a manual baseline check against a published secure configuration baseline | 0 per tenant open over 30 days | 1 to 3 per tenant open over 30 days | More than 3 per tenant, or any critical open over 30 days |
| S10 | SaaS without SSO | In-scope tenants that issue integration credentials or hold third-party grants but whose administrators and users do not sign in through the central IdP / in-scope tenants | Application inventory; IdP application list | 0% | Above 0% to 10% | Above 10% |
Notes on the formulas
- S2 needs a written scope classification: publish once which scopes count as high risk for each platform, and keep it stable. RFC 9700 asks that access token privileges be restricted to what the use case needs and to specific resources and actions; scopes outside that test are your high-risk list. The OAuth Scope Design Guide explains scope granularity.
- S3 is a signal, not a verdict: publisher verification confirms who the publisher is, not that the app is safe. Unverified apps with broad scopes are the priority because illicit consent grants often use them. See the Consent Phishing and Illicit OAuth Grants Guide.
- S4 uses activity, not age: a grant made last week and never used is dormant in intent; one used daily is not, however old. Where the platform does not log token use per app, count it as unknown and report that number too.
- S5 counts the grant, not the account: disabling a user does not always revoke tokens already issued to apps on their behalf. Test it on your platforms and record the result in the evidence pack.
- S7 follows RFC 9700: the BCP says refresh tokens for public clients must be sender-constrained or use refresh token rotation. Long-lived, unrotated refresh tokens are what made a 2025 chatbot integration token theft reach hundreds of customer CRM tenants. The OAuth Refresh Token Rotation and Expiry Guide covers the mechanics.
- S9 is integration-scoped: count only posture findings about third-party access, so this metric does not repeat a general SaaS posture review.
How to collect the data
- Fix the denominator: take the list of sanctioned SaaS tenants from the application inventory, not from what the posture tool already sees. Unsanctioned apps found later go to the Shadow SaaS and Unsanctioned App Discovery Guide process.
- Export grants per tenant: each tenant's admin console or API can list OAuth apps, connected apps or installed integrations with scopes, consent type (user or admin), consenting user, publisher status and creation date. Normalise them into one register with a common scope risk field.
- Export credentials: the tenant's list of API keys, personal access tokens and integration service accounts with owner, creation, last-used and expiry fields where the platform records them.
- Pull activity: SaaS audit logs for the last 90 days, filtered to token issue and API calls by app ID. Check retention first; some tiers keep logs for less than 90 days. See the SaaS Audit Log Retention and Licensing Guide.
- Join to HR and the IdP: match consenting users to the HR leaver extract by employee identifier (S5), and tenants to the IdP application list (S10).
- Sample and re-perform: pick five grants per tenant and trace each from export to metric result. Record any variance in the evidence pack.
Red flags that need action now
- Any unverified app holding mail, file or directory write scopes tenant-wide (S2, S3).
- An app granted in the last 30 days with high-risk scopes by a user outside IT, and no review record (S2, S6).
- Any grant still active from a leaver who held an administrator role (S5).
- A vendor breach notice for an integration you use, with no revocation and re-issue of its tokens within 24 hours. Follow the OAuth Token Theft Incident Playbook.
- Refresh tokens or API keys with no recorded issue date or older than a year on a high-scope integration (S7, S8).
- A tenant where user consent to third-party apps is fully open with no admin consent workflow (S9).
- Audit logs that cannot show which app made which API call, so S4 cannot be measured.
Findings log
| Finding ID | Finding | Metric | Severity | Owner | Due date | Status |
|---|---|---|---|---|---|---|
| HC-SAAS-01 | [Number] unverified apps with high-risk scopes in [Tenant] | S2, S3 | [High / Medium / Low] | [Role] | [Date] | [Open / In progress / Closed / Risk accepted] |
| HC-SAAS-02 | [Number] grants consented by leavers still active | S5 | [Severity] | [Role] | [Date] | [Status] |
| HC-SAAS-03 | [Number] grants dormant over 90 days in [Tenant] | S4 | [Severity] | [Role] | [Date] | [Status] |
| HC-SAAS-04 | [Number] API keys with no owner or expiry in [Tenant] | S8 | [Severity] | [Role] | [Date] | [Status] |
| HC-SAAS-05 | [Finding] | [Metric] | [Severity] | [Role] | [Date] | [Status] |
Group grants with one root cause, such as an open consent setting, into one finding. Risk accepted findings carry an expiry date and the approver's name, and are re-opened at the next quarterly run.
Remediation map
| Finding type | Metrics | Where the fix is explained |
|---|---|---|
| Incomplete grant inventory | S1 | SaaS and OAuth App Governance Guide; How to Inventory third party oauth apps in google workspace |
| Over-broad or unverified grants | S2, S3, S6 | OAuth Scope Design Guide; Consent Phishing and Illicit OAuth Grants Guide |
| Dormant and leaver grants | S4, S5 | OAuth Token Introspection and Revocation Guide |
| Old tokens and unmanaged keys | S7, S8 | OAuth Refresh Token Rotation and Expiry Guide; SaaS API Token Security Guide |
| Posture findings on third-party access | S9 | SSPM rollout playbook |
| Tenants outside SSO | S10 | SaaS Admin Account Protection |
Related guides
- SaaS and OAuth App Governance Guide: the controls this health check measures.
- SaaS Application Security Review Checklist: the review before an app is approved.
- OAuth and SaaS integration breaches: incidents behind the red flags.
- OAuth Token Theft Incident Playbook: when a finding becomes an incident.
- NHI Governance Maturity Model: place the results on a maturity level.
References
Framework anchors for the metrics: NIST SP 800-53 AC-2 (Account Management), AC-2(3) (Disable Accounts), CA-3 (Information Exchange) and SA-9 (External System Services); NIST CSF 2.0 PR.AA-05 and GV.SC-07; CIS Controls v8.1 Control 15 (Service Provider Management); OWASP Non-Human Identities Top 10 2025 NHI3 Vulnerable Third-Party NHI and NHI7 Long-Lived Secrets; MITRE ATT&CK T1528 Steal Application Access Token and T1550.001 Application Access Token.
- IETF: RFC 9700 Best Current Practice for OAuth 2.0 Security
- NIST: SP 800-53 Rev. 5
- NIST: Cybersecurity Framework 2.0
- CISA: Secure Cloud Business Applications (SCuBA) Project
- CIS: Control 15 Service Provider Management
- OWASP: Non-Human Identities Top 10 2025
- MITRE ATT&CK: T1528 Steal Application Access Token
Related NHI Mgmt Group resources: SaaS and OAuth App Governance Guide, SaaS Application Security Review Checklist, SaaS and Integration Risk Executive Briefing, OAuth Token Theft Incident Playbook.