They often mistake raw CPU or memory usage for the root cause. In Kubernetes, scaling and scheduling depend on policy thresholds, stabilization windows, and resource requests versus allocatable capacity. If those controls are mis-tuned, the platform can thrash even when utilisation appears normal.
What Kubernetes teams miss when they treat autoscaling as a utilisation problem
Autoscaling is a control-system problem, not a dashboard problem. Raw CPU or memory can be a useful signal, but it does not tell you whether pods can be scheduled, whether requests reflect reality, or whether the policy itself is causing oscillation. In Kubernetes, the important question is often whether the platform can translate demand into stable capacity without fighting its own settings.
The biggest mistake is assuming that high utilisation automatically means “scale up” and low utilisation automatically means “scale down.” Kubernetes makes decisions through multiple layers, including requests and limits, scheduler fit, HPA thresholds, stabilisation windows, and cluster-level capacity. If any of those are misaligned, the system can look healthy on paper while still thrashing, delaying placement, or overreacting to short spikes.
That means the real diagnosis starts with the control plane, not the workload chart. A deployment can fail to scale because the requests are too high for the nodes, because the autoscaler cannot act fast enough, because scale-down is too aggressive, or because the observed metric is not a good proxy for demand. Teams that only inspect utilisation often miss the control loop that is actually broken.
Why the scaling loop breaks even when metrics look normal
Kubernetes scaling depends on the relationship between desired replica count, pod requests, and allocatable node capacity. If requests are inflated, the scheduler may leave pods Pending even though nodes appear underused by application metrics. If requests are too low, the autoscaler may pack too much work onto a node and create latency before any threshold appears alarming.
Policy matters just as much as capacity. Stabilisation windows, scale-up rate limits, and scale-down delays are designed to dampen noise, but they can also hide a genuine saturation problem or create lag that makes the system feel “stuck.” The result is often oscillation, where replicas chase load too slowly on the way up and collapse too quickly on the way down.
That is why the right question is not “What is CPU right now?” but “What decision did the autoscaler make, and why did the scheduler accept or reject it?” For teams using native Kubernetes controls, NIST SP 800-190 Container Security remains a useful reference for understanding how image, runtime, and orchestrator decisions combine into platform risk. It reinforces that container behaviour is shaped by the whole system, not one metric.
What to inspect before you tune the autoscaler
Start with the metric semantics. Confirm whether the autoscaler is using CPU, memory, or a custom signal that actually tracks demand. Then compare that signal against requests, not just against node utilisation, because requests govern scheduling and often determine whether scaling pressure is real or merely visible.
Next, inspect the timing of the loop. If scale-up is slower than traffic spikes, the symptom may look like underprovisioning even when the policy is technically working as configured. If scale-down is too eager, you may see repeated churn as replicas disappear before the workload has truly settled.
Finally, review whether the cluster has enough headroom to absorb the intended behaviour of the autoscaler. An autoscaler cannot create capacity that the cluster cannot schedule, and a scheduler cannot place pods whose requests exceed what the nodes can actually offer. Teams often discover that “bad autoscaling” is really “bad sizing plus bad policy.”
In practice, the most useful external check is whether the cluster can place the replica count the policy is asking for under sustained load. If it cannot, investigate resource requests, node pool capacity, and workload topology before changing thresholds. When the issue is capacity-shaping rather than workload demand, tuning the HPA alone usually just makes the same mistake happen faster.
Risk and Threat Considerations
Poor autoscaling is an availability risk because it can create either starvation or runaway cost. Under-scale conditions lead to queue buildup, latency spikes, and failed requests, while over-aggressive scale-up or scale-down can cause instability even when average utilisation looks acceptable.
Failure mechanism: The control loop reacts to an incomplete signal, or its thresholds and delays are misaligned with workload behaviour, so the platform oscillates between too few and too many replicas.
Impact: Users see intermittent failures or slowdowns, operations teams lose confidence in autoscaling, and remediation shifts to manual intervention, which usually hides the underlying control issue instead of fixing it.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST SP 800-53 Rev 5 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | SI-13 — Predictable Failure Prevention | Autoscaling stability depends on preventing control-loop failure and oscillation. |
| CM-2 — Baseline Configuration | Replica policies, requests, and limits must be baselined to avoid drift-driven instability. | |
| Recommendation — Tune scaling controls to prevent thrash and preserve predictable service behavior. Baseline pod requests, limits, and autoscaling policies against observed workload behavior. | ||
| NIST CSF 2.0 | PR.PS-01 — Configuration Management | Autoscaling is a configuration problem when policy thresholds and delays are mis-set. |
| Recommendation — Manage autoscaling policy as a controlled configuration artifact and review changes carefully. | ||
Practitioner Guidance
What to verify: Check whether requests, limits, and autoscaler thresholds describe the same workload reality. If requests are consistently far from observed demand, treat the requests as a scaling input problem, not just a cost issue.
Decision rule: If pods are Pending or thrashing while CPU looks normal, assume the bottleneck is scheduling, policy timing, or capacity headroom until proven otherwise. Do not retune thresholds before you confirm that the replica can actually be scheduled and held steady.
What practitioners underestimate: Stabilisation settings are not just guardrails, they are part of the behaviour of the system. A “correct” threshold can still produce a bad outcome when the delay and damping values do not match traffic shape.
Practitioner takeaway: Effective autoscaling is about aligning metrics, requests, and control timing so the cluster can make stable decisions under real load. If those three disagree, the autoscaler will expose the mismatch rather than solve it.