Why Kubernetes autoscaling fails for AI workloads and how DRA fixes it

Kubernetes has long relied on a clean separation between its scheduler, which places pods on existing nodes, and its autoscaler, which adds new nodes when capacity runs out. This division of labor worked well for standard compute but is now breaking down under the weight of AI workloads that require specific accelerator topologies. As of October 2026, the community is actively redesigning these components to ensure they speak the same language about hardware constraints.

What happened

For a decade, the assumption that compute resources were largely interchangeable allowed the scheduler and autoscaler to operate independently. However, AI accelerators like GPUs are not generic; their performance depends heavily on how they are connected to CPUs, memory, and other GPUs. A common failure mode now sees pods remaining pending even when the cluster appears to have capacity. The autoscaler adds a node, but the scheduler rejects it because the hardware topology does not match the workload’s strict locality requirements.

The core issue is that the autoscaler answers whether to add a node, while the real question is whether the added node is the right shape for the work. If the autoscaler provisions a node without understanding the intricate wiring of the accelerators, the scheduler cannot place the pod. This mismatch leaves expensive accelerator capacity unused and delays critical inference or training jobs. The Cloud Native Computing Foundation (CNCF) highlights that this is not just a vendor-specific problem but a fundamental architectural challenge in upstream Kubernetes.

To address this, the community is moving toward a shared vocabulary for hardware description. Historically, Kubernetes treated accelerators as simple integer counts. Now, with the introduction of Dynamic Resource Allocation (DRA), workloads can file structured claims that describe not just how many devices they need, but how those devices must be related to each other and to network interfaces. This allows both the scheduler and DRA-aware autoscalers to evaluate capacity using the same detailed criteria.

How it works

Dynamic Resource Allocation changes the resource model from a bare count to a structured claim. Instead of requesting four GPUs, a workload specifies that it needs four GPUs located within the same NUMA domain and connected via high-bandwidth links. The scheduler reads these claims directly to make placement decisions. Crucially, DRA-aware autoscalers are beginning to use this same information when simulating potential capacity additions. This ensures that when the autoscaler decides to scale up, it selects node types that actually satisfy the topology constraints defined by the workload.

Beyond topology, DRA also enables finer-grained sharing of accelerators. Modern devices are powerful enough that assigning an entire GPU to a small job is wasteful. DRA supports partitionable devices, allowing a single physical accelerator to be split into isolated instances or consumable shares. Multiple workloads can draw measured portions of a device simultaneously. The scheduler tracks these fractional allocations, and autoscalers can weigh prospective nodes based on available slices rather than just whole devices. This reduces the all-or-nothing waste that previously plagued GPU utilization.

Despite these advances, a gap remains for gang scheduling, where a group of pods must be scheduled together or not at all. Current workload-aware scheduling APIs allow the scheduler to hold pods until a full group can be placed, but the autoscaler still reasons pod by pod. It may provision partial capacity for a few members of a gang, wasting resources while the job waits for the rest. A provisional mechanism called the provisioning request attempts to bridge this by letting workloads ask for group capacity atomically, but it remains a bolt-on solution rather than a deeply integrated feature.

Key details

• Topology matters: Accelerator performance depends on locality, including CPU NUMA domains, PCIe paths, and inter-GPU links, not just device count.
• Dynamic Resource Allocation (DRA): Replaces integer-based resource requests with structured claims that describe device relationships and network interfaces.
• Shared vocabulary: DRA allows both the scheduler and autoscaler to understand hardware constraints identically, reducing provisioning mismatches.
• Fractional sharing: DRA supports partitionable devices and consumable capacity, enabling multiple workloads to share a single accelerator efficiently.
• Gang scheduling gap: While the scheduler supports atomic group placement, the autoscaler still scales pod-by-pod, leading to potential waste for large distributed jobs.
• Future integration: SIG-Autoscaling and SIG-Scheduling are designing APIs to let the scheduler see provisionable capacity and request it explicitly.

Why it matters

For engineering teams managing AI infrastructure, this mismatch translates directly into wasted budget and slower iteration cycles. GPU hours are expensive, and having them sit idle because of a scheduling-provisioning disagreement is a significant operational cost. Understanding that a pending pod might not indicate a lack of regional capacity, but rather a topology mismatch, changes how teams debug and optimize their clusters. It shifts the focus from simply adding more nodes to ensuring that the nodes being added are compatible with the workload’s structural needs.

Furthermore, the move toward integrated design means that future Kubernetes features will consider autoscaling implications from the start. Rather than retrofitting autoscaling support after a new scheduling primitive ships, the community is building interfaces that allow the two control loops to coordinate proactively. This evolution is critical for supporting bursty inference and agentic workloads that cannot tolerate the latency of cold starts or misprovisioned nodes. Teams running accelerators now have a stake in these upstream discussions, as the resulting standards will define the efficiency of their infrastructure for years to come.

What you can do

• Audit pending pods: Check if pending GPU pods are due to topology mismatches rather than true capacity exhaustion.
• Adopt DRA: Migrate workloads to use Dynamic Resource Allocation claims to express topology and sharing requirements explicitly.
• Enable workload-aware scheduling: Test the beta Workload and PodGroup APIs to improve gang scheduling behavior, noting they are disabled by default.
• Evaluate autoscaler compatibility: Verify if your current autoscaler implementation is DRA-aware and can simulate topology constraints during scale-up.
• Monitor fragmentation: Track GPU utilization metrics to identify waste from all-or-nothing assignments and consider enabling fractional sharing.
• Engage with SIGs: Participate in SIG-Autoscaling and SIG-Scheduling discussions to influence the design of upcoming integration APIs.