Compiling an NES CPU into the weights of a Qwen3.5 language model

Developer David Breunig has released nesformer, a project that compiles the Nintendo Entertainment System’s 6502 CPU directly into the weights of a standard Qwen3.5 language model. Published in late October 2026, this work demonstrates that a large language model can execute binary code by treating its internal state as machine memory, rather than learning behavior from data.

What happened

The project builds on previous work called Qwenputer, which compiled simpler architectures like CHIP-8 and RISC-V into Qwen3.5-4B. Nesformer advances this by targeting the MOS 6502 processor, the heart of the NES. Instead of training the model on gameplay footage or assembly code, Breunig used a compiler to compute every single weight in the checkpoint. The resulting model is a standard Hugging Face checkpoint that loads with Qwen3_5ForCausalLM.from_pretrained and requires no custom code to initialize.

The repository includes a command-line interface that allows users to run NES ROMs. When executed, the model generates a token stream that represents the CPU’s execution log. A separate host program handles the peripheral hardware, such as the Picture Processing Unit (PPU) for graphics and the controller inputs. The system successfully runs test ROMs like Thwaite and Full Game, producing frame-perfect output that matches traditional emulators. It also supports Super Mario Bros., provided the user supplies the ROM file.

How it works

The core mechanism relies on the Gated DeltaNet architecture within the Qwen3.5 model. In this setup, the model’s internal state matrix functions as the computer’s random-access memory (RAM). Each row in the 128x128 state matrix acts as a memory cell. By using one-hot keys and queries, the model can overwrite specific rows to write data or read them back to retrieve values. This allows the 2 KB of work RAM, the 32 KB of program ROM, and the CPU registers to reside entirely within the model’s weights and state.

The compilation process maps the 6502’s logic gates and opcodes into the model’s layers. The circuit spans seven Gated DeltaNet layers. The first layer anchors the program counter and stores flags, while subsequent layers handle instruction fetching, address decoding, and arithmetic operations. The final layer predicts the next token in the execution trace. Because the weights are computed mathematically from the circuit design, they are sparse: out of 466.6 million parameters, only 9.3 million are nonzero, allowing the checkpoint to compress to just 3.3 MB.

The host program manages everything outside the CPU. It injects tokens for non-maskable interrupts (NMI) at vblank and provides input data when the CPU reads from I/O registers. To manage speed, the host detects idle loops where the CPU waits for vblank and skips the redundant iterations, advancing the clock directly to the next event. This optimization reduces the token count significantly, though the system still runs roughly 150 to 290 times slower than real hardware.

Key details

• The model uses the Qwen3.5-4B architecture with 7 Gated DeltaNet layers and a hidden size of 768.
• Only 9.3 million of the 466.6 million parameters are nonzero, resulting in a 3.3 MB compressed checkpoint.
• The system supports NROM cartridges (iNES mapper 0) but does not yet support bank-switching mappers like MMC1 or MMC3.
• Execution speed ranges from 10 tokens per second with stock transformers to 1,650 tokens per second with a numba-compiled engine.
• The host program handles the PPU rendering and controller input, injecting necessary tokens into the model’s context.
• Unofficial opcodes and jumps into RAM cause the emulation to halt, as the circuit only implements official 6502 instructions.

Why it matters

For AI engineers and researchers, nesformer provides a concrete example of how transformer internals can be repurposed for deterministic computation. It challenges the assumption that large models must be trained on vast datasets to perform complex tasks. By compiling logic directly into weights, developers can create models that guarantee specific behaviors, offering a path toward more verifiable and transparent AI systems. This approach separates the "thinking" part of the model from the "state" management, clarifying how attention mechanisms can mimic memory access.

For software developers interested in emulation, this project highlights the trade-offs between accuracy and performance. While the model produces bit-perfect results for supported games, the computational overhead is immense compared to traditional emulation. However, the ability to run executable code inside a standard LLM checkpoint opens new avenues for integrating legacy software logic into modern AI pipelines without requiring external interpreters or virtual machines.

What you can do

• Install the required dependencies, including torch>=2.6 and transformers>=5.22, to run the checkpoint with stock libraries.
• Use the --show flag in the CLI to render game frames directly in your terminal if it supports 24-bit color.
• Run the check command to compare the model’s output against a reference CPU implementation and verify correctness.
• Supply your own NROM .nes files, such as Donkey Kong or Excitebike, to test compatibility with different games.
• Experiment with the --buttons script option to automate controller inputs for testing specific game sequences.
• Explore the nesformer/spec.py file to understand the token families and how each CPU instruction is encoded.