Tagged "attention-optimization"

4 articles tagged attention-optimization, 14 April 2026 to 27 October 2026. Newest first.

• llama.cpp release b11927: Metal backend gains GGML_OP_TOP_K_MASK optimization 27 October 2026

  llama.cpp release b11927 brings a fused top-k mask operation to the Metal backend, porting optimizations from CUDA to improve inference performance on Apple Silicon devices. The grid-stride kernel implementation speeds up attention operations critical to LLM performance.

• llama.cpp Enables Sparse Flash Attention for Qwen4 with CUDA Optimization 20 September 2026

  The latest llama.cpp release adds sparse flash attention support for Qwen4 models on CUDA hardware, improving inference efficiency and throughput for locally deployed LLMs.

• llama.cpp 0.4.0 Released with Sparse Flash Attention and RDMA Support 5 September 2026

  llama.cpp 0.4.0 introduces major performance improvements including sparse flash attention, RDMA support, Qwen3.8-Flash-Next support, on-demand tensor reading, and upgraded GGML 0.23.0, enabling more efficient local inference at scale.

• oMLX Framework Implements DFlash Attention for Optimized Inference 14 April 2026

  The oMLX framework has added DFlash attention implementation, improving inference efficiency on local hardware. This update represents progress in core optimization techniques for on-device LLM execution.