Why MoE Models Benefit More from Speculative Decoding
Cohere Blog2 days agoBenchmarkCohere analyzes why speculative decoding behaves differently on Mixture-of-Experts models than on dense LLMs. Its benchmarks show MoE speedups can peak at moderate batch sizes because sparse expert routing keeps verification bandwidth-bound. The post also finds that temporal expert overlap and fixed overhead amortization make multi-token verification cheaper than simple worst-case models predict.
NVIDIA Releases NVFP4-Quantized DiffusionGemma 26B A4B IT on Hugging Face
r/LocalLLaMA top day3 days agoReleaseNVIDIA has released DiffusionGemma 26B A4B IT NVFP4 on Hugging Face, a quantized version of Google DeepMind's open-weights multimodal model. Built on a Mixture-of-Experts architecture with 25.2B total but only 3.8B active parameters, it generates text in parallel 256-token blocks using discrete diffusion, exceeding 1,100 tokens per second on H100 hardware. The model supports a 256K-token context, text/image/video inputs, native function calling, reasoning mode, and 35+ languages.
Introducing Mellum2: A 12B Mixture-of-Experts Model by JetBrains
Hugging Face Blog13 days agoReleaseJetBrains introduced Mellum2, a 12B Mixture-of-Experts model. The supplied title confirms the model name, publisher, scale, and architecture description only. Without the article body, its intended use, licensing, availability, training details, benchmarks, and deployment requirements cannot be verified.