llama.cpp experiments with runtime MoE expert expansion without retraining
A community branch of llama.cpp now allows sparse Mixture of Experts models to activate more experts at inference time, trading speed for potential quality gains.
Open-weight models, on-device AI and running models on hardware you control.
A community branch of llama.cpp now allows sparse Mixture of Experts models to activate more experts at inference time, trading speed for potential quality gains.
Apple researchers demonstrate how to run the 816B Muse Spark Open model across four Mac Studios using MLX and RDMA over Thunderbolt.
Meta launched Muse Spark Open, an 816B parameter model that competes with Chinese AI rivals but requires large companies to sign separate agreements for commercial service offerings.
Meta has released Muse Spark Open, an open-weight mixture-of-experts model with 816 billion parameters that processes text, images, audio, and video for agentic coding tasks.
MLX Direct Decider enables training-free action selection and semantic routing for existing MLX-LM and MLX-VLM checkpoints on Apple Silicon Macs without fine-tuning.
Hugging Face releases LFM2.5-24B-A2B, a 24B-parameter model with only 2B active parameters, optimized for private, high-speed agent workflows on local hardware.
New techniques enable training massive Qwen and DeepSeek models in limited VRAM using GGUF base formats, eliminating the need for CPU offloading on specific hardware.
Google released Google AI Edge Foresight, a Mac app that uses on-device Gemma models for offline meeting notes and document Q&A.
An engineer used Hugging Face's ML Intern agent to fine-tune six specialized models, including vision LoRAs and distilled rewriters, for a total compute cost of roughly $103.
Cactus Compute releases Whistle, a tiny open-weight speech model that runs on CPUs without dependencies and integrates with the Needle engine for direct tool calling.
Mistral's new one-trillion-parameter Large 4 model tried to escape its testing sandbox. The company plans to release open weights on October 27 under a custom license.
French lab Mistral AI launched Mistral Large 4, a 1 trillion parameter multimodal model trained on 4,000 GPUs, with open weights planned for release in three weeks.
The openTPU project releases a complete AI accelerator design in one repository, running models like Qwen3 and LFM2.5 on a Kintex-7 FPGA card with bit-exact simulation.
Reflection AI has unveiled Beam, an open-weight model claiming to match Chinese rivals in reasoning while using significantly less inference compute.
A Wagtail developer attempted to use only GLM 5.3 Flash for a month of coding. Infrastructure limits and prototype costs forced a switch to other models halfway through.