Tagged "compiler-optimization"
6 articles tagged compiler-optimization, 7 April 2026 to 13 October 2026. Newest first.
-
SonicBoom: llama.cpp's Native Kernels Deliver 14% Decode Speed Improvements
llama.cpp introduces SonicBoom, a graph compiler with custom kernels that achieves 14% faster token generation, addressing one of the most critical bottlenecks in local LLM inference.
-
Modular's MAX Inference Stack Achieves 5.5x Speedup on AMD Hardware
Modular's Dakkak reports MAX Inference delivering 5.5x faster performance on Kimi K2.5, with AMD's 30% hardware discount reshaping the economics of local model deployment.
-
Qualcomm Acquires Modular AI in $3.9 Billion Deal to Accelerate On-Device AI
Qualcomm's acquisition of AI software startup Modular signals a major push to optimize LLM deployment on mobile and edge devices. The deal aims to enhance Qualcomm's compiler and runtime technology for efficient on-device inference.
-
AMD Unveils Ryzen AI Halo Developer Platform for On-Device AI Workloads
AMD releases the Ryzen AI Halo developer platform and Ryzen AI Max PRO 400 series processors specifically optimized for on-device AI inference. These processors target enterprise and consumer deployments of local language models with dedicated neural processing capabilities.
-
Lython: Experimental Python Compiler Toolchain Based on LLVM
Lython offers an experimental Python compiler leveraging LLVM, potentially enabling faster execution of Python-based inference workloads. This tool demonstrates emerging approaches to optimizing performance in local model deployment.
-
TurboQuant-Optimized llama.cpp Fork Delivers GFX906 GPU Acceleration
Community developer releases optimized llama.cpp fork featuring TurboQuant quantization and specialized GFX906 GPU optimizations with Gemma 4 architecture support coming soon.