NVIDIA DGX Spark scales to eight nodes for local frontier AI models

NVIDIA has expanded its desktop AI hardware strategy by officially supporting eight-node clusters for the DGX Spark system. Announced at GTC Berlin in October 2026, this update scales unified memory to 1 TB, enabling developers to run massive frontier-class models locally. The release includes new software optimizations for collective communication and multi-model orchestration, aiming to bring large-scale infrastructure patterns to individual workstations.

What happened

The core advancement is the ability to link eight DGX Spark units into a single coherent cluster. Each unit provides 128 GB of unified memory, so an eight-node setup delivers 1 TB of total addressable memory. This capacity allows local execution of models that previously required cloud-based data centers, such as the full GLM 5.3 (753B) parameter model from Z.ai. NVIDIA states that this configuration can serve the GLM 5.3 Flash variant at approximately 80 tokens per second, making interactive development with frontier architectures feasible on-premises.

To manage these clusters, NVIDIA Sync will include Cluster Assistant support for device management and setup by the end of the year. The hardware relies on low-latency ConnectX-7 network interface cards and RDMA over Converged Ethernet (RoCE) to handle tensor exchanges between nodes. This networking foundation supports both switch-connected topologies for flexible workload partitioning and direct ring connections for cost-effective setups.

How it works

A key software innovation is RoCEnante, a Spark-optimized collective backend designed to reduce latency during tensor-parallel decode operations. In standard ring all-reduce algorithms, data passes sequentially between neighboring ranks, which can introduce latency for small messages. RoCEnante uses an all-to-all exchange where each rank shares input directly with every other rank, followed by local reduction. This approach minimizes communication steps for modest rank counts, though it increases total data movement. It is merged into FlashInfer and falls back to the standard NVIDIA Collective Communications Library (NCCL) if startup tests fail.

For workload management, NVIDIA NeMo Switchyard enables multi-model orchestration on shared clusters. A lightweight classifier, such as OpenBMB’s MiniCPM5 2B, inspects incoming requests and routes them to appropriate model pools. Simple tasks go to smaller, faster models, while complex queries are directed to larger ones like Qwen3.8 Flash Next. This routing happens over switch-connected clusters, which provide the any-to-any connectivity required for independent model groups. The system uses a YAML-based policy file, allowing developers to tune traffic splits based on their specific needs.

Key details

• Eight DGX Spark nodes provide 1 TB of unified memory, enabling local execution of the 753B parameter GLM 5.3 model.
• The RoCEnante backend delivers a 4.43x decode speedup at eight nodes compared to 3.05x without it, specifically for switch-connected topologies.
• Decode throughput for Qwen3.8-27B scales from roughly 12 tokens per second on one node to 52 tokens per second on eight nodes.
• NeMo Switchyard can increase aggregate throughput up to 3x by routing most traffic to smaller models while maintaining quality on tool-use benchmarks.
• Pre-training a nanochat model across eight nodes achieves a 7.79x speedup with 97.4% scaling efficiency, demonstrating near-linear performance.
• Physical ring topologies offer a lower-cost option for four-node inference, while switches are required for eight-node clusters or split allocations.

Why it matters

For software engineers and technical founders, this development shifts the boundary of what is possible on local hardware. Running a 753B parameter model locally eliminates data egress costs and latency associated with cloud APIs, while also addressing privacy concerns for sensitive datasets. The ability to scale memory to 1 TB means that teams can experiment with mixture-of-experts architectures and large context windows without provisioning remote instances. This democratizes access to frontier models, allowing for rapid iteration and debugging in a controlled environment.

The introduction of optimized collective backends and multi-model orchestration tools also highlights a maturing ecosystem for distributed local computing. Developers no longer need to manually manage complex networking stacks or write custom routing logic to balance cost and performance. Tools like Switchyard and RoCEnante abstract these complexities, enabling teams to focus on model behavior and application logic rather than infrastructure tuning. This reduces the operational overhead of maintaining high-performance AI clusters on-premises.

What you can do

• Review the model compatibility guide to identify which frontier models fit within the 1 TB memory limit of an eight-node cluster.
• Set up a two-node DGX Spark cluster using the provided networking guide to test distributed workloads before scaling up.
• Implement NeMo Switchyard with a sample YAML policy to route requests between different model sizes based on complexity.
• Benchmark your current inference workloads using vLLM across multiple Spark nodes to measure throughput gains from tensor parallelism.
• Explore the nanochat pre-training code on GitHub to understand how near-linear scaling applies to training small models from scratch.
• Evaluate whether a switchless ring topology meets your needs for four-node deployments to reduce hardware costs.