Tagged "long-context-inference"

4 articles tagged long-context-inference, 28 March 2026 to 4 August 2026. Newest first.

• LLM Memory Doesn't Only Get Written Wrong, It Goes Wrong Later 4 August 2026

  Research on how LLM memory degrades and becomes corrupted over time during inference. Understanding memory behavior is critical for reliable local deployment.

• Exploiting Sparsity for Long Context Inference: Million Token on Commodity GPUs 10 July 2026

  A new technique enables million-token context windows on standard consumer GPUs by leveraging sparsity optimizations. This breakthrough makes long-context LLM inference practical and affordable for self-hosted deployments.

• Gemma 4 31B vs Qwen 3.5 27B: Comprehensive Long Context Benchmark 11 April 2026

  Community benchmark comparing Gemma 4 31B and Qwen 3.5 27B for long context workloads on 24GB VRAM, establishing these as the top local models for mid-range GPU setups.

• TurboQuant KV Cache Compression Achieves 22.8% Faster Decoding at 32K Context 28 March 2026

  Google's TurboQuant compression method has been successfully integrated into llama.cpp, enabling 4.6x KV cache compression and 22.8% decode speedup at 32K context length by skipping 90% of dequantization work. This breakthrough makes long-context inference practical on consumer hardware like MacBook Air M4.