Tagged "long-context-window"

3 articles tagged long-context-window, 28 March 2026 to 6 June 2026. Newest first.

• Running Infinite Context Lengths on 8GB GPU Without Out Of Memory 6 June 2026

  A new engine enables running LLMs with effectively infinite context windows on consumer GPUs with just 8GB VRAM by avoiding memory exhaustion. This breakthrough makes long-context inference practical for edge and local deployments.

• TurboQuant KV Cache Compression Achieves 22.8% Faster Decoding at 32K Context 28 March 2026

  Google's TurboQuant compression method has been successfully integrated into llama.cpp, enabling 4.6x KV cache compression and 22.8% decode speedup at 32K context length by skipping 90% of dequantization work. This breakthrough makes long-context inference practical on consumer hardware like MacBook Air M4.

• Qwen3 512k Context via TurboQuant on Mac mini 28 March 2026

  Qwen3 achieves 512k token context window using TurboQuant quantisation on Mac mini hardware, demonstrating significant advances in local long-context model deployment.