NVIDIA Snapshot cuts GPU workload startup from minutes to seconds

NVIDIA has released Snapshot, an open-source checkpoint and restore system designed to accelerate GPU workloads on Kubernetes. Published on November 10, 2026, this tool captures the fully initialized state of CPU and GPU memory, allowing inference replicas to start in seconds rather than minutes. It targets the bottleneck of cold starts in large language model serving, where initializing weights and compiling kernels often delays scaling.

What happened

The core problem addressed by Snapshot is the latency involved in bringing new inference replicas online. When demand spikes or a node fails, infrastructure teams must provision new pods. This process, known as a cold start, involves loading container images, starting the inference engine, reading model weights into GPU memory, compiling code with tools like torch.compile, and capturing CUDA graphs. For large models, this sequence can take several minutes, during which available GPUs sit idle instead of serving requests.

Snapshot changes this workflow by treating the initialized workload as a reusable artifact. Once a pod is fully ready, Snapshot captures its entire memory state. When a new replica is needed, the system restores this state directly onto compatible hardware. This bypasses the heavy lifting of weight loading and compilation. The system supports complex parallelism strategies, including tensor parallelism, data and expert parallelism, and decode context parallelism. It handles checkpoints up to 1 TB across eight GPUs, making it suitable for large mixture-of-experts models.

Early adopters have reported significant improvements. Crusoe, a contributor to the project, reduced the readiness time for GLM-5.3 pods from approximately eight minutes to 31 seconds on NVIDIA HGX B300 hardware. Similarly, Saturn Cloud cut vLLM startup times from a range of two to 8.5 minutes down to six to 34 seconds. These results demonstrate that the technology is viable for production environments requiring rapid scale-from-zero capabilities.

How it works

Snapshot operates through two primary actions: checkpoint and restore. Both are managed via Kubernetes custom resources. To create a checkpoint, users define a PodSnapshot resource for a running pod, or a SnapshotJob to checkpoint a pod as soon as it becomes ready. The system pauses the workload briefly to capture the CPU and GPU memory state into a portable artifact. This artifact includes the initialized model weights, compiled kernels, and runtime buffers, but excludes the key-value cache, which is cleared before capture.

Restoration is triggered by adding a specific annotation, nvidia.com/restore-from, to a new pod specification. The Snapshot agent on the target node locates the artifact and begins the restore process. This involves using Checkpoint/Restore in Userspace (CRIU) to reload the CPU process state and a specialized CUDA restore mechanism to copy GPU state back into video memory. The workload then performs a quick handshake to adapt to the new node and resume serving. This process skips the initial engine startup, weight loading, and graph capture phases entirely.

Key details

• Restore times range from 3.6 to 42.7 seconds across tested configurations, representing an 8x to 36x speedup over cold starts.
• Checkpoint creation takes between 15 seconds and 5.5 minutes, depending on model size, but this cost is amortized over multiple restores.
• The combined time for checkpointing and restoring is shorter than a single cold start in every tested configuration.
• For large eight-GPU mixture-of-experts models, restoring GPU state accounts for 18 to 29 seconds of the total restore time.
• CPU process restoration via CRIU takes 4.2 to 10.8 seconds, indicating that GPU state transfer is the dominant factor in large restores.
• The system supports vLLM 0.31.0, SGLang 0.5.21, and NVIDIA TensorRT LLM 1.3.0 on NVIDIA B200 GPUs with VAST NFS storage.

Why it matters

For engineering teams building AI infrastructure, the gap between provisioning a GPU and serving traffic is a major efficiency leak. Cold starts force teams to over-provision capacity to handle sudden spikes, as they cannot rely on rapid scaling. By reducing startup time to under 45 seconds, Snapshot enables more aggressive autoscaling policies. This means clusters can remain leaner during low-demand periods and expand quickly when traffic surges, improving overall resource utilization and reducing cloud costs.

Furthermore, this technology simplifies recovery from failures. If a node goes down, replacing the lost capacity no longer requires waiting minutes for new pods to initialize. The restored pods are immediately ready to accept requests, maintaining service level agreements more effectively. This is particularly critical for large mixture-of-experts models, where initialization complexity grows with model size. The ability to checkpoint and restore terabyte-scale states makes managing these massive models more operationally feasible.

What you can do

• Visit the NVIDIA Snapshot repository to access installation instructions and usage guides for your Kubernetes cluster.
• Review the NVIDIA Dynamo user guide to learn how to integrate Snapshot with inference workers for automated restoration.
• Test the provided examples for vLLM, SGLang, and TensorRT LLM to evaluate compatibility with your current stack.
• Ensure your hardware meets the requirements, specifically matching GPU models and drivers between checkpoint and restore nodes.
• Join the NVIDIA Dynamo Slack workspace and participate in the #snapshot-dev channel to discuss integration challenges.
• Benchmark your specific workloads to determine the optimal checkpoint frequency based on the 15-second to 5.5-minute creation window.