Tagged "gguf-quantization"
7 articles tagged gguf-quantization, 14 February 2026 to 13 July 2026. Newest first.
-
Show HN: GGUFun, Play Snake and a Simple Maze on Ollama Using Hand Crafted GGUFs
A creative demonstration of running game logic directly on Ollama using custom GGUF quantized models. This shows innovative approaches to local inference beyond traditional language understanding tasks.
-
Unsloth Completes Comprehensive MiniMax M2.7 GGUF Quantization Suite
Unsloth has finished quantizing MiniMax M2.7 across the full range of GGUF quantization levels from 1-bit to BF16, providing practitioners with optimized variants for every hardware configuration from edge devices to high-end systems.
-
Quantization Strategy Comparison: Balancing Quality and Speed on Consumer Laptops
Detailed benchmarking of different GGUF quantization methods for Qwen 3.5 4B on Intel Lunar Lake iGPU reveals optimal compression strategies for small model deployment on resource-constrained hardware.
-
Google Gemma 4 Released with GGUF Quantizations
Google has released Gemma 4 with multiple model sizes (26B, 31B variants) already quantized in GGUF format by Unsloth, enabling immediate local deployment on consumer hardware.
-
Coding Implementation to Run Qwen3.5 Reasoning Models Distilled With Claude-Style Thinking Using GGUF and 4-Bit Quantization
A new implementation enables running distilled Qwen3.5 reasoning models with 4-bit quantization and GGUF format, making advanced reasoning capabilities accessible on consumer hardware. This combines distillation, quantization, and standardized formats for practical local deployment.
-
Qwen3.5-35B Unsloth Dynamic GGUFs Achieve SOTA Across Nearly All Quantisation Levels
New state-of-the-art GGUF quantisations for Qwen3.5-35B released with 150+ KL Divergence benchmarks and 9TB of variants. Critical tool calling chat template bug fixed affecting all quantisation uploaders.
-
MiniMax-M2.5 230B MoE Model Released with GGUF Support for Local Deployment
MiniMax-M2.5, a 230B parameter mixture-of-experts model, is now available in GGUF format for local deployment with impressive performance benchmarks on consumer hardware.