Use vertical pod autoscaling when the main constraint is CPU or memory per pod and the workload is best served by resizing existing instances. Use horizontal pod autoscaling when demand is better handled by adding more replicas. For gateway workloads, the right choice depends on whether throughput limits are driven by per-pod resources or by the need to distribute traffic across more pods.
How to choose the scaling pattern for an API gateway
For api gateway workloads, the first question is whether the bottleneck is inside each pod or across the service as a whole. If a single gateway pod runs hot on CPU or memory, vertical pod autoscaling is the cleaner fit because it adjusts the size of existing pods. If traffic needs more parallel capacity, horizontal pod autoscaling is the better match because it adds replicas and spreads request load.
That distinction matters because gateways are usually state-light but latency-sensitive. A configuration that improves per-pod headroom can still leave the service under-provisioned at peak request volume, while a pure replica increase can fail to help if each pod is already resource constrained or if the gateway depends on tight per-pod memory limits for stability.
What changes with gateway behaviour, not just generic Kubernetes advice
API gateways sit at a traffic choke point, so the scaling signal should reflect how the gateway actually fails under pressure. If latency rises because request parsing, routing tables, TLS handling, or policy evaluation consumes too much CPU per instance, resizing a pod may restore headroom without changing the service topology. If the gateway is limited by concurrent request volume, connection fan-out, or per-instance saturation, more replicas usually give a better result.
It is also worth separating compute pressure from application-level limits. Some gateways become inefficient when they hold large in-memory caches or perform heavy header and token inspection, which makes vertical scaling more effective. Others are designed to distribute traffic evenly and recover better from replica growth, especially when they are stateless enough for load balancing to do the real work.
In practice, teams often need to ask which failure mode they are trying to avoid: a pod OOM or CPU throttle event, or a cluster-wide throughput ceiling. The answer determines whether the control loop should resize the pod or increase the replica count, and that decision should be based on observed saturation patterns rather than on preference for one autoscaling model.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | PR.PT — Protective Technology | Gateway scaling is an availability control choice that shapes how service capacity is maintained. |
| PR.AC — Access Control | API gateways enforce access decisions, so scaling must preserve stable request handling. | |
| Recommendation — Tune autoscaling to preserve service availability under peak gateway demand. Keep gateway access enforcement reliable while scaling capacity. | ||
| CIS Controls v8 | 11 — Data Recovery | Resilience planning for gateway capacity depends on recovery and continuity under load failure. |
| Recommendation — Validate gateway scaling choices against recovery and continuity requirements. | ||
Practitioner Guidance
What to verify: Check whether gateway latency, throttling, or error rates correlate more closely with per-pod resource pressure or with aggregate request volume. If pod-level metrics show repeated CPU throttling or memory pressure before traffic saturation, vertical scaling is the first lever; if each pod stays healthy but total traffic exceeds service capacity, horizontal scaling is the safer choice.
- Watch for mixed signals, such as pods that are both resource constrained and replica constrained, because that usually means autoscaling alone is not the full fix.
- Prefer the simplest control that matches the dominant bottleneck, then confirm that the gateway still has enough headroom for spikes, deployments, and failover.
Decision rule: If the gateway keeps failing inside the pod, resize it; if the gateway stays healthy but cannot absorb more concurrent demand, add replicas.
Practitioner takeaway: The right autoscaling model is the one that matches the bottleneck you can measure, not the one that sounds more elastic in theory.
Related resources from NHI Mgmt Group
- How should security teams decide whether JIT access is safe for non-human identities?
- How should security teams decide between a lightweight gateway and a full identity provider for self-hosted apps?
- How should platform teams govern Kubernetes-native API gateway resources?
- How should security teams decide between gateway-level control and container isolation for agents?
