Meta releases Muse Spark Open, an 816B parameter multimodal MoE model

Meta has officially released Muse Spark Open, a frontier-scale open-weight model designed for complex multimodal tasks. Published on October 15, 2026, this mixture-of-experts architecture represents the most capable US open-weights model to date, targeting developers who need advanced reasoning across code, media, and long contexts.

What happened

Muse Spark Open arrives as the full-size counterpart to Muse Glimmer, which Meta released in August 2026. While Glimmer is a smaller dense model suitable for local execution, Spark Open scales up significantly to handle enterprise-grade workloads. The model features approximately 816 billion total parameters, yet it activates only around 42 billion parameters per token during inference. This design allows it to maintain high performance while managing computational costs more effectively than traditional dense models of similar size.

The primary focus of this release is agentic coding. Meta trained the model on real developer workflows, including multi-file refactors, long debugging sessions, and tool usage. Unlike previous iterations, Muse Spark Open integrates text, images, audio, and video into a single interface. This means developers can feed the model screenshots, voice notes, or screen recordings alongside their codebase, allowing the AI to plan, act, observe, and self-correct over extended tasks. It also supports web browsing with image inputs, expanding its utility beyond static code analysis.

How it works

The core innovation in Muse Spark Open is its Mixture-of-Experts (MoE) architecture. The language model consists of 62 layers, each containing a hidden dimension of 8192. Instead of using a single dense feed-forward network in each layer, the model employs an MoE block with 256 distinct experts. A router mechanism selects only eight of these experts for each token processed. Each expert is a feed-forward network that projects data from 8192 to 4096 dimensions and back, applying its own normalization. Although all 256 experts must remain in memory, requiring the storage footprint of an 816-billion-parameter model, the computational load per token matches that of a 42-billion-parameter model. This structure holds about 97% of the total model parameters.

To handle non-text inputs, the model uses specialized encoders. The perception encoder, with 1.8 billion parameters, processes images and video frames by splitting them into 14x14 patches. It uses sparse attention in most layers, with full attention applied every fourth layer and at the final layer. For audio, a separate tokenizer converts sound into tokens at a rate of one every 80 milliseconds, or 12.5 tokens per second. These audio tokens are marked in the chat template and fed into a convolutional pyramid encoder that processes temporal frames through various patch sizes.

The model supports a context window of up to one million tokens, which includes text, image, video, and audio tokens combined. To manage the massive key-value cache required for such a long context, Muse Spark Open uses grouped-query attention with 128 query heads and 16 key-value heads. In BF16 precision, the cache consumes approximately 250 KB per token, totaling about 266 GB for a full million-token context. The model is available in two checkpoint formats: BF16, which requires four nodes of eight H200 GPUs with tensor parallelism, and NVFP4, a 4-bit precision format optimized for NVIDIA Blackwell GPUs that can run on a single B200 node.

Key details

• Model Scale: Approximately 816 billion total parameters with 42 billion active parameters per token.
• Architecture: 62-layer Mixture-of-Experts with 256 experts per layer, selecting 8 experts per token.
• Context Window: Supports up to 1,048,576 tokens, encompassing text, images, audio, and video.
• Input Modalities: Accepts text, images, audio (12.5 tokens/second), and video via specialized encoders.
• Checkpoint Formats: Available in BF16 (approx. 1.5 TB) and NVFP4 for NVIDIA Blackwell GPUs.
• Inference Requirements: BF16 requires 32-way tensor parallelism across H200 GPUs; NVFP4 runs on a single B200 node.

Why it matters

For software engineers and technical leads, Muse Spark Open offers a unified interface for multimodal development tasks. The ability to process screenshots and screen recordings alongside code allows for more intuitive debugging and documentation generation. By training specifically on agentic workflows, Meta has created a model that can autonomously navigate complex codebases, reducing the manual effort required for large-scale refactoring. The open-weight nature of the release also means organizations can fine-tune the model on proprietary codebases without relying on closed API services, enhancing data security and customization.

However, the infrastructure requirements present a significant barrier to entry. Running the BF16 checkpoint demands substantial GPU resources, specifically multiple nodes of H200 GPUs. Even the optimized NVFP4 version requires access to next-generation Blackwell hardware. This limits immediate adoption to well-funded enterprises or research institutions with access to cutting-edge compute clusters. Developers must carefully evaluate their hardware capabilities before attempting to deploy or fine-tune this model, as the memory and processing demands are far beyond typical local development environments.

What you can do

• Download the Muse-Spark-Open-816B-A42B or NVFP4 checkpoints from the official repository.
• Use the provided llama.cpp, vLLM, or SGLang recipes to set up inference environments.
• Configure the MuseSparkProcessor to handle multimodal inputs including images, audio, and video.
• Experiment with the 1M token context window for long-document summarization or extensive codebase analysis.
• Fine-tune the model on specific coding tasks using the distributed configuration settings for tensor and expert parallelism.
• Evaluate the NVFP4 checkpoint if you have access to NVIDIA Blackwell GPUs to reduce hardware requirements.