llama.cpp experiments with runtime MoE expert expansion without retraining
A community branch of llama.cpp now allows sparse Mixture of Experts models to activate more experts at inference time, trading speed for potential quality gains.
Results for “llama.cpp”
Daily coverage of AI, developer tools and infrastructure. Each story explains what happened and why it matters, with a link to the original source.
Magnitude: An open source inference engine that tunes itself to your hardware
Magnitude compiles and tunes kernels on-device to run open models up to 2x faster than llama.cpp across Apple Silicon, NVIDIA, AMD, and CPU setups.
