Google Gemma 4 Released with GGUF Quantizations

Google's new Gemma 4 model has arrived with immediate community support for local deployment. The Unsloth quantizations provide GGUF-format variants optimized for llama.cpp inference, making the 26B and 31B models accessible on mid-range consumer hardware.

Early testing shows strong performance characteristics for local deployment. Users report Gemma 4 26B achieves around 81 tokens/sec on Apple Silicon (M5 MAX) with efficient VRAM utilization, and demonstrates improved reasoning capabilities compared to earlier Qwen 3.5 variants. The model includes thinking tokens for chain-of-thought reasoning, opening possibilities for complex local inference tasks.

This release is significant for the local LLM community as it provides a modern, well-quantized option directly competitive with proprietary models, with immediate availability in formats suitable for edge and self-hosted deployment.

Source: r/LocalLLaMA · Relevance: 9/10