Open & local AI

Inkling 2 Mini brings looped transformer architecture to local hardware

Thinking Machines Lab releases Inkling 2 Mini, a multimodal mixture-of-experts model designed for local setups with a unique looped layer architecture.

A glass cube with looping internal layers representing the Inkling 2 Mini architecture.
Illustration generated for this article

Thinking Machines Lab has released Inkling 2 Mini, a new multimodal mixture-of-experts model optimized for local deployment. Published on November 20, 2026, this model introduces a looped transformer architecture that balances parameter efficiency with high performance, targeting developers who need capable AI without datacenter-scale infrastructure.

What happened

Inkling 2 Mini is a significant departure from standard dense or traditional sparse models. It features approximately 26 billion physical parameters but activates only 3 billion per token. The model achieves this efficiency through a novel structural trick: it loops through its middle layers three times. This design allows the model to behave as if it had 48 logical layers while physically storing only 24. The result is a system that maintains high capacity and reasoning depth while keeping the memory footprint small enough for local machines.

The release includes support for both text and image inputs, with a context window of up to 1 million tokens. Thinking Machines Lab ships the model in mixed-precision formats, specifically using NVFP4 and MXFP8 for expert weights, which helps reduce storage and memory requirements further. A DSpark drafter is also included to enable speculative decoding, offering additional speed improvements at a minimal memory cost. The model is positioned as a direct competitor to other mid-sized local models, with early tests suggesting performance comparable to Qwen 3.8 27B.

How it works

The core innovation in Inkling 2 Mini is its looped execution path. The model consists of 24 decoder layers, but the processing flow is not linear. Layers 0–5 and 18–23 are visited once, while the middle block, layers 6–17, is executed three times in a loop. Crucially, these three passes reuse the same weights. To prevent the model from simply repeating itself, it uses cross-pass messages. Each pass reads information from the previous pass’s outputs via learned queries for key, value, and MLP inputs. A gating controller then decides how much of the new input to keep and how much of the previous message to integrate, allowing each loop to refine the computation.

This architecture is supported by a mixture-of-experts mechanism. At each layer visit, the router selects 8 experts from a pool of 128. The hidden size is 2,304, with an intermediate expert width of 1,152. The attention mechanism uses 24 query heads and eight KV heads, incorporating learned relative position logits and dynamic convolutions for full-attention layers. This combination of weight sharing, iterative refinement, and sparse expert selection allows the model to punch above its weight class in terms of logical depth and parameter count.

Key details

  • Architecture: Looped transformer with 24 physical layers acting as 48 logical layers via triple-looping of the middle 12 layers.
  • Parameters: ~26 billion total physical parameters, with 3 billion active parameters per token.
  • Context Window: Supports up to 1 million tokens for both text and image inputs.
  • Precision: Ships in mixed-precision format with NVFP4 and MXFP8 expert weights.
  • Performance: Leads in 32 out of 48 benchmarks in the Artificial Analysis Intelligence Index v5, including coding and vision tasks.
  • Inference Support: Available via Transformers, Llama App (llama.cpp), MLX, and SGLang with day-zero support.

Why it matters

For software engineers and technical founders, Inkling 2 Mini represents a shift toward more efficient local AI. The ability to run a model with the logical depth of a 48-layer system on hardware that can only store 24 layers reduces the barrier to entry for sophisticated AI applications. This is particularly relevant for coding assistants, agentic workflows, and vision-based tools that require long context windows but must operate within strict memory constraints. The inclusion of mixed-precision weights and speculative decoding support means that developers can achieve higher throughput without needing expensive GPU clusters.

The model’s performance in benchmarks like FrontierCode v2.0 and Terminal-Bench 4.0 suggests it is well-suited for development environments. By leading in many categories against larger models like Gemma 4 31B and Nemotron 3.5 Super VL, Inkling 2 Mini offers a compelling alternative for teams looking to deploy capable AI locally. The availability of a research variant with variable loop passes also opens doors for experimentation, allowing users to trade off speed for quality depending on their specific use case.

What you can do

  • Test the model locally: Use Llama App or llama.cpp with the provided GGUF quants (IQ4_NL or Q8_0) to run Inkling 2 Mini on macOS or Windows.
  • Integrate with Python: Use the transformers library and AutoModelForMultimodalLM class to load the model and process text or image inputs.
  • Experiment with loop passes: Try the TML-d24-varloop-research checkpoint to adjust the number of loop passes and observe the impact on speed and output quality.
  • Deploy with SGLang: Set up a production-ready server using SGLang, ensuring you run it on SM103 hardware for optimal performance.
  • Enable speculative decoding: Add the DSpark drafter to your SGLang setup to boost generation speed, especially for long-context tasks.
  • Use MLX for Apple Silicon: Leverage the upstreamed MLX support to run the model natively on Apple hardware with NVFP4 and MXFP8 quantization.
  • #ai
  • #local-llm
  • #transformers
  • #machine-learning

Tools from the Bytechap store

$89

DocBento

Self-hosted document management that reads every scan and answers with page citations.

Live demo

Keep reading

All stories