Splitting an application into microservices turns one trust boundary into hundreds, and every internal call becomes an API that can be reached, replayed or abused. A monolith has one front door and in-process function calls; a microservices system has an edge gateway, a mesh of service-to-service calls, message queues, sidecars, service discovery and a short-lived workload for every request burst. Controls that worked at the perimeter do not follow the traffic inside. This guide covers service identity, authentication and authorisation between services, user identity propagation, segmentation, secrets, logging and resilience, with the NIST SP 800-204 series and the OWASP cheat sheets as the main references. It is written for application security engineers, platform teams and architects who design or review microservices platforms, on Kubernetes, serverless or virtual machines.
Key takeaways
- Every service enforces its own authorisation. The gateway rejects obviously bad requests, but OWASP is explicit that each service must check access to its protected operations, including calls from other internal services.
- Give each workload a cryptographic identity. Short-lived X.509 certificates issued per workload (for example SPIFFE X.509-SVIDs) replace shared API keys and IP allow lists as the basis for service-to-service trust.
- Keep the user and the calling service separate. Propagate the end user's context in a form each recipient can validate, authenticate the calling service independently, and use token exchange rather than widening token audiences.
- Default-deny the network as well as the API. Mutual TLS in strict mode plus network policies stop a compromised pod from reaching services and metadata endpoints it never needed.
- Design logging for the call chain. A correlation ID on every hop, structured logs, and secrets stripped before a log line is written are what make an incident in a distributed system investigable.
What changes when you move to microservices
NIST SP 800-204 frames microservices security around a set of core features that every such system needs: authentication and access management, service discovery, secure communication protocols, and availability techniques such as load balancing, throttling and circuit breakers. It names two frameworks that bundle these features, the API gateway and the service mesh. SP 800-204A then gives deployment guidance for proxy-based service meshes, and SP 800-204B covers attribute-based access control (ABAC) inside a mesh, describing zero trust as mutual authentication between any two services.
| Concern | Monolith | Microservices | Security consequence |
|---|---|---|---|
| Trust boundary | One perimeter, in-process calls | Every service interface is a network API | Internal APIs need authentication and authorisation, not just the edge |
| Identity | User session in memory | User context and caller service identity travel across hops | Tokens can be replayed, forged or over-scoped between services |
| Network | Few flows, easy to firewall | Dynamic IPs, autoscaling, east-west traffic | IP-based rules break; workload identity and labels replace them |
| Secrets | A handful of database credentials | Credentials per service, per environment, per dependency | Secret sprawl in images, environment variables and config maps |
| Logging | One log file per tier | Logs scattered across ephemeral containers | Evidence lost unless shipped centrally with a correlation ID |
Granularity matters for security too: too coarse and one flaw exposes unrelated functions, too fine and identities and flows multiply. A practical rule is to draw service boundaries around data ownership: one service owns a data store, and other services reach that data only through its API, never by sharing database credentials.
Service identity: the foundation
Service-to-service trust has to rest on something stronger than "the request came from inside the cluster". The options, in rough order of strength:
| Approach | How it works | Weaknesses | When it fits |
|---|---|---|---|
| Network location (IP allow lists, "internal" VPC) | Trust any caller from an internal address range | Any compromised pod inherits trust; breaks with autoscaling | Never sufficient on its own |
| Shared static API keys | Caller sends a long-lived secret in a header | Leaks via logs and images, hard to rotate, no per-instance identity | Legacy integrations during migration only |
| Signed tokens from a security token service | Caller obtains a short-lived signed token carrying its ID and scopes | Bearer token can be replayed if intercepted; local validation cannot see revocation before expiry | Calls crossing L7 proxies or third-party gateways |
| Mutual TLS with per-workload certificates | Both sides present certificates from a trusted CA; identity is in the certificate | Needs PKI, trust bootstrap, rotation and revocation | Default for east-west traffic inside a platform |
| mTLS plus tokens | mTLS proves the calling workload; a token carries the user or delegated context | More moving parts | Sensitive services acting on behalf of users |
The OWASP Microservices Security Cheat Sheet lists the main challenges of mTLS as key provisioning and trust bootstrap, certificate revocation and key rotation. That is the problem SPIFFE was designed for:
- SPIFFE ID: a URI of the form
spiffe://<trust domain>/<workload identifier>, for examplespiffe://acme.com/billing/payments. SPIFFE recommends separate trust domains for different security practices, such as staging and production. - SVID: the document that proves the identity. The X.509-SVID is the recommended form; the JWT-SVID is a fallback because an intercepted JWT can be replayed, and it is mainly needed when an L7 proxy or load balancer terminates TLS between two workloads.
- Workload API: delivers the SPIFFE ID, private key, short-lived certificate and trust bundle to the workload without the workload holding any bootstrap secret, and rotates keys automatically before expiry.
- Trust bundle: the set of CA roots a workload uses to verify peers, also retrieved and refreshed through the Workload API.
Service meshes apply the same idea. Istio, for example, derives a workload principal from the Kubernetes service account in the form cluster.local/ns/<namespace>/sa/<service-account> and rotates workload certificates automatically through its agent and istiod. One service account per service, never the namespace default account, is therefore a precondition. See the SPIFFE and SPIRE Guide, the mTLS Guide and the Cloud Workload Identity Guide.
Where identity fits: users, services and delegation
Most requests inside a microservices system carry two identities: the end user (or external client) who started the request, and the service making the current hop. Confusing the two is the root of many authorisation flaws. The OWASP Identity Propagation Patterns Cheat Sheet describes four patterns:
| Pattern | What travels downstream | Main risk | Use when |
|---|---|---|---|
| Token forwarding | The external access token, validated by each recipient | Every service that sees the bearer token is a place it can leak; pressure to widen the audience | Small systems where every recipient is an intended audience |
| Service-signed identity forwarding | User attributes in a header or JSON object signed by the forwarding service | Any trusted service can misrepresent the user; a signature proves the sender, not entitlement | Avoid, or limit to an authenticated edge proxy that strips client-supplied copies |
| Token exchange (RFC 8693) | A new token from a security token service for a specific downstream audience | Adds a dependency on the token service | Downstream call needs a different audience or explicit delegation |
| Trusted internal assertion | A normalised internal token issued after the edge validates the external credential | Assertion issuer becomes a high-value target; limit lifetime and recipients | Large platforms with many external protocols |
- Token exchange in practice: RFC 8693 defines the grant type
urn:ietf:params:oauth:grant-type:token-exchange. The service sends the incoming token assubject_token, can authenticate itself with anactor_token, and names the target withaudienceorresource. The issued token can carry anactclaim identifying the current actor, which preserves the difference between delegation (service A acting for user B) and impersonation (A indistinguishable from B). See the OAuth Token Exchange and Delegation Guide. - Validate every token fully: signature, allowed algorithm, issuer, audience, token type and validity period. Never accept an OpenID Connect ID token as an API access token. See the JWT Security Guide.
- Choose validation mode deliberately: introspection (RFC 7662) reflects revocation but adds latency and a dependency; local validation of JWT access tokens (RFC 9068 profile) is fast but cannot see revocation before expiry, so it needs short lifetimes. Bound introspection cache time and never cache beyond token expiry. If validation cannot complete, reject the request.
Service identities are non-human identities with the same lifecycle problems as any other: ownership, least privilege, rotation and decommissioning. See the NHI Authentication Guide.
Authorisation: edge, service and object level
The OWASP cheat sheet separates two layers, and both are required.
- Edge-level authorisation: the API gateway authenticates the external caller, rejects requests without valid credentials, applies coarse scope checks and rate limits, and must not be bypassable. If a service can be reached directly, gateway checks are decoration.
- Service-level authorisation: each service checks that this caller (user and calling service) may perform this operation on this resource. Object-level, tenant-level and business rules belong here, because only the service has the resource context. Broken object level authorisation (OWASP API1:2023) and broken function level authorisation (API5:2023) are the typical result when this layer is missing. See the Top 10 API Security Risks.
- Mesh-level service authorisation: a service mesh can enforce which workloads may call which services and methods. In Istio,
AuthorizationPolicyactions are evaluated in the order CUSTOM, DENY, ALLOW, with DENY taking precedence over ALLOW. Note thatRequestAuthenticationrejects invalid JWTs but accepts requests with no token at all unless an authorisation rule requires one, a common gap. - Policy as code: as the number of services grows, OWASP recommends shared, reviewed policies rather than hand-written checks in each codebase. SP 800-204B describes ABAC in the mesh, combining subject, resource and environment attributes. External policy decision points (for example an OPA or Cedar engine) need a defined behaviour when the policy service is unavailable: fail closed for sensitive operations.
For the trade-offs between RBAC, ABAC and relationship-based models see the Authorisation Models Guide.
Gateways, routing and segmentation
East-west segmentation limits what a compromised service can reach. It works at two layers that complement each other: network policy at layers 3 and 4, and identity-based policy at layer 7 in the mesh.
- Kubernetes NetworkPolicy: pods are non-isolated until a policy selects them, policies are additive (there is no deny rule), and they only work if the CNI plugin enforces them. Start each namespace with a default-deny policy for ingress and egress (
podSelector: {}), then allow named flows by label, remembering that a default egress deny also blocks DNS until port 53 to the cluster DNS is allowed. See the Kubernetes Security Guide and the Network Segmentation Guide. - Mesh mTLS mode: Istio
PeerAuthenticationsupports STRICT (mTLS only), PERMISSIVE (mTLS and plaintext) and DISABLE. A mesh-wide policy with the mode unset behaves as PERMISSIVE, which is useful during migration and dangerous if left in place. Plan the move to STRICT per namespace. See the Service Mesh Security Guide. - Egress control: restrict outbound traffic to known destinations through an egress gateway or allow list. This is what blocks a server-side request forgery (OWASP API7:2023) from reaching a cloud metadata endpoint or an internal admin API. See the SSRF Prevention Guide.
- Gateway hardening: one gateway per trust zone (public, partner, internal) rather than one gateway with every route; strip client-supplied identity headers at the edge; keep route definitions in version control and review them, because a new route is a new exposed API. See the API Security Guide.
Secrets and configuration
- Remove the secrets you can: workload identity (SPIFFE, cloud workload identity federation, managed identities) replaces many static credentials for calls to cloud APIs and other services. See the Workload Identity Federation Guide.
- Inject the rest at runtime: pull database passwords and third-party API keys from a secrets manager at start-up or through a sidecar, scoped to the one service that needs them. Never bake them into images, where they persist in layers. See the Secrets Management Guide.
- Protect signing keys: the keys of the internal token service and the mesh CA can mint any identity. Keep them in an HSM or managed KMS, rotate them on a schedule, and monitor issuance.
Logging, monitoring and observability
The OWASP logging architecture for microservices is a useful baseline: each service writes structured logs to stdout or a local file, a logging agent on the host forwards them to a message broker (such as Kafka or NATS), and a central service consumes them. Specific requirements from the cheat sheet:
- Correlation ID: generate one per call chain and include it in every log message so a request can be traced across services.
- Secrets out at source: exclude secrets and unnecessary sensitive data before the service writes the log entry; filtering at the agent cannot remove data already written locally.
| Signal | Source | What it can reveal |
|---|---|---|
| Authorisation denials per service and caller | Service logs, mesh access logs | Probing for object IDs, a compromised service calling unusual peers |
| Plaintext connections in a mesh | Mesh telemetry | Workloads outside mTLS, PERMISSIVE mode left on |
| New service-to-service edges | Mesh or flow logs compared with a baseline graph | Lateral movement, unreviewed integrations |
| Token validation failures | Gateway and service logs | Replayed, expired or forged tokens, wrong audience |
| Requests to metadata or admin endpoints | Egress gateway, DNS and flow logs | SSRF attempts and credential theft |
Relevant ATT&CK techniques for detection engineering include T1190 Exploit Public-Facing Application, T1552.005 Cloud Instance Metadata API, T1528 Steal Application Access Token and T1550.001 Application Access Token (replay of stolen service tokens).
Resilience and abuse controls
SP 800-204 places load balancing, throttling and circuit breakers alongside authentication as core features, because in a distributed system an availability failure in one service cascades.
- Rate limits at two levels: per client at the gateway, and per calling service inside the mesh, so a compromised or buggy internal service cannot exhaust a shared dependency (OWASP API4:2023 Unrestricted Resource Consumption). See the Rate Limiting and API Abuse Guide.
How to implement: a phased approach
- Inventory (weeks 1 to 4): list every service, its owner, its exposed routes, its data stores and its credentials. Build the actual call graph from mesh or flow logs, not from architecture diagrams.
- Identity (weeks 3 to 8): give every service its own service account and workload identity. Deploy a mesh or SPIRE to issue short-lived certificates. Enable mTLS in PERMISSIVE mode first and watch telemetry for plaintext callers.
- Enforce mTLS (per namespace): switch to STRICT one namespace at a time, starting with the least critical. Roll back by setting the PeerAuthentication mode back to PERMISSIVE.
- Network default deny: add default-deny NetworkPolicies per namespace with allow rules generated from the observed call graph. Test in a staging cluster with production-like traffic, then apply to production with the previous policy set ready to restore.
- Service authorisation: add mesh AuthorizationPolicies in a dry-run or audit mode where your mesh supports one, review the would-be denials, then enforce. Add object-level checks in services, starting with those holding personal or financial data.
- Identity propagation: replace forwarded external tokens with token exchange or internal assertions, audience-restricted per service. Remove service-signed identity headers.
- Secrets clean-up: scan images and repositories, move remaining secrets to a vault, rotate anything that was ever in an image or environment variable.
- Measure and test: add the metrics below to a platform dashboard and include service-to-service abuse cases (direct calls bypassing the gateway, cross-object access, SSRF) in penetration tests.
Small teams and large platforms
- Small teams (under 20 services): a managed mesh or the cloud provider's workload identity plus a hardened gateway and NetworkPolicies covers the essentials. Running a self-hosted PKI and SPIRE may cost more than it returns.
- Large or multi-cluster platforms: SPIFFE trust domains per environment and federation between them, central policy as code, and a platform team that owns the mesh CA and identity issuance. Treat the platform's identity plane as tier-zero infrastructure.
- Serverless and managed services: there may be no sidecar. Use the platform's native identity (function execution roles) with narrowly scoped permissions and signed requests, and apply the same object-level checks in code.
Measuring success
- mTLS coverage: percentage of east-west requests over authenticated mTLS (target 100% for production namespaces).
- Default-deny coverage: percentage of namespaces with default-deny ingress and egress policies.
- Static credential count: number of long-lived service-to-service secrets remaining, trending down.
- Services with object-level authorisation tests: percentage with automated negative tests for cross-object access.
- Trace completeness: percentage of requests whose logs carry a correlation ID across every hop.
Common failure patterns
- Trusting the internal network: services that accept any request from inside the cluster turn one exploited service into access to everything. The Capital One breach shows the pattern at cloud scale: a request forgery through a web application firewall reached the instance metadata service, returned the firewall role's temporary credentials, and that over-broad role could read customer data in storage. Egress control, IMDSv2 and a narrowly scoped workload role each would have broken the chain.
- Authorisation only at the gateway: the gateway checks a scope, and the service assumes anything that reaches it is allowed, so any internal caller or gateway bypass gets full access.
- PERMISSIVE mode forever: mTLS is "enabled" in a dashboard, but plaintext is still accepted, so an attacker simply does not use TLS.
- Forwarding the user's token everywhere: one wide-audience token accepted by every service means any service that leaks it, through logs or a compromised dependency, leaks access to all of them.
- Shared service accounts and credentials: several services share one Kubernetes service account or database user, so the mesh cannot tell them apart and logs cannot attribute actions.
- Secrets in logs and images: debug logging of request headers captures bearer tokens; credentials in image layers outlive the rotation of the original.
Framework mapping
| Control area | NIST SP 800-53 Rev. 5 | NIST CSF 2.0 | MITRE D3FEND / ATT&CK |
|---|---|---|---|
| Service identity and mutual authentication | IA-9 Service Identification and Authentication; SC-8 Transmission Confidentiality and Integrity; SC-23 Session Authenticity | PR.AA-05 | D3-CBAN Certificate-based Authentication; D3-MAN Message Authentication |
| Service and object authorisation | AC-3 Access Enforcement; AC-6 Least Privilege; AC-24 Access Control Decisions | PR.AA-05 | Counters T1550.001 Application Access Token |
| Segmentation and egress | SC-7 Boundary Protection; AC-4 Information Flow Enforcement | PR.IR-01 | D3-NI Network Isolation; counters T1552.005 |
| Signing keys and certificates | SC-12 Cryptographic Key Establishment and Management | PR.DS-01 | Counters T1528 |
| Logging and monitoring | AU-2 Event Logging; AU-12 Audit Record Generation | PR.PS-04; DE.CM-01 | Detection for T1190 |
| Resilience | SC-5 Denial-of-service Protection | PR.IR-01 | Counters T1499 Endpoint Denial of Service |
In ISO/IEC 27001:2022 Annex A the closest controls are A.8.27 Secure system architecture and engineering principles, A.8.22 Segregation of networks, A.8.5 Secure authentication, A.8.15 Logging and A.8.26 Application security requirements. In CIS Controls v8.1, Control 16 (Application Software Security), Control 12 (Network Infrastructure Management) and Control 3 (Data Protection) are the relevant ones.
Practitioner checklist
- Every service has its own service account and workload identity, and no workload runs under a namespace default account.
- East-west traffic in production uses mTLS in STRICT mode, with plaintext connections alerted on.
- Every namespace has default-deny ingress and egress NetworkPolicies, with allow rules matching the reviewed call graph.
- Each service enforces object-level and function-level authorisation, with automated negative tests.
- Requests without a token are explicitly rejected where a token is required, not just requests with invalid tokens.
- User context is propagated by token exchange or internal assertions with per-service audiences; ID tokens are never accepted as access tokens.
- No service-to-service static secrets are baked into images or environment variables; remaining secrets come from a vault.
- Egress to cloud metadata endpoints and the internet is blocked except for named destinations.
- Every log line carries a correlation ID and no bearer tokens or secrets.
- Mesh CA and token-service signing keys are in a KMS or HSM, rotated, and their issuance is monitored.
Standards and references
- NIST: SP 800-204 Security Strategies for Microservices-based Application Systems
- NIST: SP 800-204A Building Secure Microservices-based Applications Using Service-Mesh Architecture
- NIST: SP 800-204B Attribute-based Access Control for Microservices-based Applications using a Service Mesh
- OWASP: Microservices Security Cheat Sheet
- OWASP: Identity Propagation Patterns Cheat Sheet
- IETF: RFC 8693 OAuth 2.0 Token Exchange
- SPIFFE: SPIFFE Concepts
- Istio: Security Concepts
Related NHI Mgmt Group resources: API Security Guide · Cloud Workload Identity Guide · Kubernetes Security Guide · Authorisation Models Guide · Top 10 API Security Risks