Google's Gemma 4 Just Got 3x Faster - MTP Drafters Land for the Open Models

Google has released MTP (multi-token prediction) drafters that accelerate Gemma 4 inference by up to three times without modifying the core models. The system uses speculative decoding, where smaller drafter models predict upcoming tokens whilst the main Gemma 4 model verifies these predictions in parallel. The approach maintains Gemma 4's reasoning capabilities whilst reducing latency through parallelisation rather than sequential processing. When the drafter's predictions align with what Gemma 4 would generate, the entire sequence is accepted in a single pass, significantly speeding up inference. The performance gains are particularly notable for the 26B MoE and 31B Dense Gemma 4 variants running on consumer hardware. For edge deployment, the faster inference translates to improved battery life on mobile devices, addressing a key constraint for local AI applications. The MTP drafters are available under the same Apache 2.0 licence as the core Gemma models, with immediate support across major inference frameworks including Hugging Face Transformers, MLX, vLLM, SGLang, and Ollama. This marks another step in making speculative decoding standard practice for optimising large language model inference without retraining.