Most API breaches do not start at a misconfigured gateway route; they start at an API nobody listed, an endpoint that was never tested for object level authorisation, or a version that stayed live because no one owned its retirement. This health check measures the API security programme across every API the organisation runs or exposes: internal and external, behind a gateway or not, built in house or published by a partner integration. It gives nine programme metrics with formulas, thresholds and evidence so an application security lead can report the same figures every quarter. It is written for application security leads, API product owners, internal audit and the CISO's office.
This page is the programme view. It does not repeat the per-route gateway configuration metrics (route policies, rate limit settings, client key age, backend TLS) which sit in the API gateway health check. Build-time controls are in the API Security Checklist; this review asks whether those controls are applied to every API and keep holding over time.
What this health check covers
- Scope: every production API and API version in the organisation's estate, including internal service APIs, mobile and single page application back ends, partner and B2B APIs, and APIs exposed by SaaS platforms the organisation configures. APIs that bypass the gateway are in scope, which is exactly why the gateway figures alone are not enough.
- Who runs it: the application security lead collects the figures; API product owners accept findings for their APIs; the head of engineering approves remediation plans; the CISO or delegate signs off red items.
- Standards basis: the metrics follow the risks in the OWASP API Security Top 10 2023, chiefly API1 Broken Object Level Authorization, API2 Broken Authentication, API3 Broken Object Property Level Authorization and API9 Improper Inventory Management, and the lifecycle view in NIST SP 800-228, which recommends controls for both the pre-runtime and runtime stages of an API.
- Search terms it answers: "API security KPIs", "API security audit", "how many shadow APIs do we have" and "how to measure API security testing coverage".
Key takeaways
- Inventory is the denominator. Every other figure is a percentage of the API inventory, so a weak inventory makes every other metric look better than it is.
- Count shadow APIs, not just inventoried ones. Discovery from traffic, code and cloud accounts is the only way to measure what the inventory misses.
- Verify by test, not by design. Authentication and authorisation coverage here is what testing proved, not what the design document says.
- Authorisation is the gap. Object level authorisation (BOLA) needs its own coverage figure because scanners and gateways rarely catch it.
- Thresholds are a starting point. The green, amber and red bands are NHI Mgmt Group recommendations; agree your own and keep them stable so trends mean something.
How to use this template
Copy the metrics table and findings log into your tracker. Fix one measurement date per quarter and take every figure as at that date. Record the numerator, denominator and data source for each metric the first time, then reuse them unchanged, so a quarter-on-quarter change reflects the estate rather than a new counting method. Every amber or red result becomes a finding with an owner and a due date. Report the absolute count beside every percentage: 96% coverage of 2,000 APIs still leaves 80 untested.
Cadence and sign-off
| Review | Frequency | Owner | Approver | Evidence kept |
|---|---|---|---|---|
| Discovery reconciliation (shadow API count) | Monthly | Application security lead | API product owners | Discovery output, inventory diff, triage decision per new API |
| Full programme metrics review (this page) | Quarterly | Application security lead | CISO or delegate | Metrics table, raw exports, findings log |
| Security test coverage review | Quarterly, aligned with the release calendar | Application security testing lead | Head of engineering | Test plan, test results per API and version |
| Independent sample test | Annually | Internal audit or external tester | Audit committee | Sample of APIs re-tested for authentication and object level authorisation |
| Event-driven review | After an API incident, acquisition or platform migration | Application security lead | CISO or delegate | Before and after figures for metrics 01 to 04 |
Metrics
NHI Mgmt Group recommended thresholds and formulas; adjust to your risk appetite. "API" means a distinct production API and major version; "sensitive API" means one that handles personal, payment, health or authentication data, or performs write operations on business records.
| ID | Metric | Formula | Data source type | Green | Amber | Red |
|---|---|---|---|---|---|---|
| APIP-01 | API inventory coverage | Production APIs found by discovery that have an inventory entry with named owner, data classification, exposure (internal, partner, public) and lifecycle state ÷ all production APIs found by discovery | API inventory or catalogue joined to discovery output | 95% or more | 85% to below 95% | Below 85% |
| APIP-02 | Shadow API count | Count of production APIs or endpoints found by discovery this period with no inventory entry; report new, triaged and still open separately | Traffic based discovery, code repository scanning, cloud account and DNS enumeration | 0 open older than 30 days | 1 to 5 open older than 30 days | More than 5, or any sensitive API open older than 30 days |
| APIP-03 | Authentication coverage (verified) | Non-public APIs where testing in the last 12 months confirmed that every operation rejects unauthenticated and expired credential requests ÷ all non-public APIs | Security test results, DAST or API test reports | 98% or more | 90% to below 98% | Below 90%, or any sensitive API failing |
| APIP-04 | Object level authorisation (BOLA) test coverage | Sensitive APIs whose operations taking an object identifier were tested in the last 12 months with a second user or tenant attempting to access the first user's objects ÷ all sensitive APIs | Penetration test and authorisation test reports, test case repository | 90% or more | 70% to below 90% | Below 70% |
| APIP-05 | Specification coverage | Production APIs with a machine readable specification (for example OpenAPI) in source control that matches the deployed version ÷ all production APIs | Specification repository compared with discovery or contract test results | 90% or more | 70% to below 90% | Below 70% |
| APIP-06 | Security testing cadence | APIs with a security test (automated in the pipeline or manual) against the current major version within the cadence set by their risk tier ÷ all production APIs | Pipeline test records, test plan, inventory risk tier | 95% or more | 80% to below 95% | Below 80% |
| APIP-07 | Sensitive data exposure findings | Count of open findings where a response returned data properties the client did not need or was not entitled to (OWASP API3) or sensitive data appeared in URLs or logs; report age of the oldest | Test findings, response schema review, data discovery in logs | 0 open past SLA | 1 to 3 open past SLA | More than 3 past SLA, or any exposing personal data in production |
| APIP-08 | API finding remediation within SLA | API security findings closed within the SLA for their severity in the period ÷ API security findings due in the period | Vulnerability or findings tracker filtered to APIs | 90% or more | 75% to below 90% | Below 75% |
| APIP-09 | Retirement plan coverage for deprecated versions | Deprecated API versions with a published sunset date, a named owner and a client migration list ÷ all deprecated API versions in the inventory | Inventory lifecycle field, deprecation notices, client registry | 100% | 90% to below 100% | Below 90% |
APIP-09 measures whether the programme has a plan for every deprecated version. Whether those versions still receive traffic at the gateway is API-07 in the API gateway health check; report both lines together rather than duplicating the traffic count here.
Why these nine and not others
- Inventory and shadow APIs (01, 02): OWASP API9 Improper Inventory Management describes how undocumented hosts and old versions become the way in. The difference between shadow and zombie APIs is set out in Shadow APIs vs Zombie APIs.
- Verified authentication and BOLA (03, 04): API1 and API2 are the top two OWASP API risks. A gateway policy shows authentication is configured; only a test shows that the back end enforces object ownership. See BOLA vs BFLA.
- Specification and testing cadence (05, 06): a current specification is what makes automated testing, schema checks and contract tests possible. NIST SP 800-53 Rev. 5 SA-11 Developer Testing and Evaluation is the control this evidences.
- Exposure and remediation (07, 08): coverage without remediation is a list of known holes; these two lines show whether findings close.
- Retirement plans (09): zombie APIs exist because nobody owned the sunset date.
How to collect the data
- Freeze the date: agree the measurement date and take every export within one working day of it.
- Run discovery from at least three angles: traffic (gateway, load balancer, service mesh and CDN logs), code (route definitions and specification files in repositories) and infrastructure (cloud API services, DNS records, exposed hosts). One source alone undercounts.
- Reconcile with the inventory: join discovery output to the inventory by host, base path and version. Unmatched discoveries feed APIP-02; matched entries missing owner, classification or lifecycle state reduce APIP-01. The API Inventory Template lists the fields.
- Pull test evidence per API: from pipeline results, DAST, manual test reports and penetration test reports, record for each API the date of the last authentication test, last authorisation test with two identities, and last full security test against the current version.
- Compare specifications with reality: diff the specification in source control against discovered endpoints; an endpoint seen in traffic but absent from the specification fails APIP-05 for that API.
- Export findings: filter the findings tracker to APIs, compute age and SLA status, and tag findings that relate to data exposure for APIP-07.
- Sample the reds: for each red metric, re-check five to ten APIs by hand to confirm the figure before it goes to the CISO.
Where identity fits
Every API client is an identity. Partner integrations, mobile back ends and service to service calls authenticate with API keys, OAuth client credentials or workload certificates, and BOLA tests need at least two test identities per tenant. Ownerless clients and long-lived keys are measured at the gateway; at programme level, make sure each inventory entry names the non-human identities that call it. See the API Key Management Guide.
Red flags that need action now
- A sensitive API discovered in traffic with no inventory entry and no owner.
- Any sensitive API never tested for object level authorisation with a second identity.
- An API that returns full records and relies on the client to filter fields (OWASP API3).
- Deprecated versions with no sunset date, especially old mobile app versions that still call them.
- Findings on sensitive APIs open past SLA for two consecutive quarters.
- Discovery limited to the gateway: APIs that bypass the gateway are the ones most likely to be missing.
Findings log
| ID | Finding | Metric | Severity | Owner | Due date | Status |
|---|---|---|---|---|---|---|
| HC-APIP-01 | [n] production APIs discovered with no inventory entry | APIP-02 | [Red / Amber] | [Application security lead] | [Date] | [Open / In progress / Closed] |
| HC-APIP-02 | [n] sensitive APIs with no BOLA test in 12 months | APIP-04 | [Severity] | [API product owner] | [Date] | [Status] |
| HC-APIP-03 | [n] APIs whose specification does not match deployment | APIP-05 | [Severity] | [API product owner] | [Date] | [Status] |
| HC-APIP-04 | Version [v] deprecated with no sunset date | APIP-09 | [Severity] | [API product owner] | [Date] | [Status] |
| HC-APIP-05 | [Finding] | [Metric ID] | [Severity] | [Owner] | [Date] | [Status] |
Escalate a finding that stays red for two consecutive quarters to the CISO, and one that repeats three times to the risk register.
Remediation map
| Finding type | Where to fix it |
|---|---|
| Inventory gaps, shadow and zombie APIs | OWASP API Top 10 API9 Improper Inventory Management Guide |
| Object level authorisation failures | OWASP API Top 10 API1 Broken Object Level Authorization Guide |
| Excessive data in responses | OWASP API Top 10 API3 Broken Object Property Level Authorization Guide |
| Testing cadence and coverage gaps | API Security Testing Plan Template |
| Route policy, rate limit and gateway key issues | API gateway health check |
| Programme maturity overall | API Security Maturity Model |
Framework mapping
| Metric | NIST CSF 2.0 | NIST SP 800-53 Rev. 5 | OWASP API Top 10 2023 |
|---|---|---|---|
| APIP-01, APIP-02 | ID.AM-02 | CM-8 | API9 |
| APIP-03, APIP-04 | PR.AA-05 | AC-3, SA-11 | API1, API2 |
| APIP-05, APIP-06 | ID.RA-01 | SA-11, RA-5 | API8 |
| APIP-07, APIP-08 | ID.RA-01, ID.IM-01 | RA-5, CA-5 | API3 |
| APIP-09 | ID.AM-02 | CM-8 | API9 |
References
- OWASP: API Security Top 10 2023
- NIST: SP 800-228 Guidelines for API Protection for Cloud-Native Systems
- NIST: SP 800-55 Vol. 1 Measurement Guide for Information Security
- NIST: SP 800-53 Rev. 5 Security and Privacy Controls
- NIST: Cybersecurity Framework 2.0
Related NHI Mgmt Group resources: API Security Guide, API Security Checklist, API gateway health check, API Security Maturity Model.