Kubernetes 1.37 moved HorizontalPodAutoscaler scale-to-zero to beta and enabled it by default. For workloads driven by suitable object or external metrics, an HPA can now reduce replicas to zero and bring them back when demand returns. That sounds like an obvious cost win: stop paying for idle workers. The reality is more useful and more nuanced.
The feature is strongest for queue consumers, batch processors, scheduled analytics helpers, and expensive workers that can tolerate startup delay. It is a poor fit for an HTTP service that must answer immediately unless another layer buffers requests. Kubernetes Services do not hold traffic while no Pods are ready, and CPU or memory cannot wake a workload when no Pod exists to report those metrics.
What changed in Kubernetes 1.37
The Kubernetes project explains that an HPA using object or external metrics can set minReplicas to zero. A queue-length or backlog metric exists independently of the worker Pods, so the autoscaler can continue observing it at zero. A new status condition distinguishes a workload automatically scaled to zero from one manually paused by an operator.
The beta feature reduces the need for an additional scale-to-zero controller in qualifying cases. That simplifies ownership, but it does not automatically create a reliable system. You still need a metrics adapter, a durable work source, correct scaling behavior, readiness, resource capacity, and a recovery path when the metric disappears.
Find workloads that are genuinely idle
Start with billing and utilization data. Identify workloads that sit at one or more replicas for long periods with no useful work. Estimate the cost of reserved CPU, memory, accelerators, licenses, and nodes attributable to that idle floor. Then estimate wake-up frequency and duration.
Do not claim the entire Pod request as savings if the node remains powered for other workloads. Real savings depend on whether node autoscaling can eventually remove unused capacity, whether specialized hardware can be released, and whether cluster commitments remain fixed. Measure at workload and infrastructure levels.
Cold start is a business decision
Wake-up time includes metric detection, the HPA control loop, scheduling, image pull, storage attachment, initialization, dependency connection, cache warm-up, and readiness. Test the full path at the least favorable time, not only with a warm image on an empty development cluster.
Define a queue-delay service objective. If work can wait sixty seconds, scale-to-zero may be attractive. If an operator expects an answer in two seconds, keep warm capacity or use a buffering front end that communicates delay honestly. Configure downscale stabilization so short quiet periods do not cause constant shutdown and restart.
Use the right waking signal
Kubernetes documentation limits scale-to-zero to object and external metrics because resource metrics require running Pods. Queue depth alone may also be insufficient. A backlog of ten ten-minute jobs is different from ten one-second jobs. Consider queue age, arrival rate, estimated work, or a composite signal exposed through a supported adapter.
Make the metric highly available. If the adapter cannot return it, the HPA reports a failed scaling condition. Alert on this state, queue age, and zero replicas with outstanding work. Define a manual scale-up procedure that responders can execute without debugging the entire telemetry stack.
Capacity must exist when demand returns
Scaling a Deployment from zero does not guarantee a node is ready. General-purpose workers may schedule quickly into spare capacity; GPU or high-memory workers may wait for node provisioning. Coordinate HPA behavior with node autoscaling and cloud quotas. Pre-pull large images where practical, reduce startup work, and use priority and limits carefully.
For factory or edge workloads, connectivity and local response requirements can make zero inappropriate. A cloud-based batch processor for archived telemetry may scale to zero safely. A local alarm or production-control dependency should not. Keep deterministic and time-critical functions in the correct operational layer.
Roll out with guardrails
Confirm the cluster version and that both API server and controller manager support the feature. Kubernetes warns about version skew and downgrade: before disabling the feature or moving to a version without the implementation, raise affected minimum replicas and scale zeroed workloads back up. Treat this as part of upgrade and rollback planning.
Begin with a noncritical queue. Set a conservative maximum, stabilization behavior, startup and readiness probes, and a Pod disruption policy appropriate to the job. Load-test from zero, burst the queue, break the metric adapter, exhaust node capacity, and observe recovery. Verify that work is idempotent because queue redelivery and worker termination can occur.
Calculate net savings
Compare idle resources removed, nodes released, accelerator hours avoided, and operational simplification against metric infrastructure, extra startup compute, longer job latency, engineering effort, and incident risk. Track time at zero, wake-up latency percentiles, queue-age percentiles, failed scale-ups, cold-start errors, and actual infrastructure cost.
Scale-to-zero is not a universal optimization. It is a precise new tool for work that can wait safely. Applied to the right queues, it can remove a stubborn idle-cost floor. Applied blindly, it converts cloud savings into user delay and operational noise.
Sources
Build it with Cogniquaint experts
Cogniquaint’s cloud platform experts can identify suitable workloads, model real savings, implement external metrics and resilient queues, tune cold starts and node capacity, and validate Kubernetes scale-to-zero through controlled failure and load testing.
Work with Cogniquaint
Ready to elevate your operations with AI-powered insights?
Get in touch with us to build your next intelligent solution.
