Tagged "llm-performance"
7 articles tagged llm-performance, 14 February 2026 to 12 May 2026. Newest first.
-
AMD's vLLM-ATOM Plugin Supercharges DeepSeek-R1 and Kimi-K2 Inference on MI350/MI400
AMD has released a vLLM-ATOM plugin optimizing inference for DeepSeek-R1, Kimi-K2, and gpt-oss-120B models on Instinct MI350 and MI400 accelerators, delivering significant performance gains for local deployment.
-
DFlash Doubles Token Generation Speed of Qwen3.5 27B on Mac M5 Max
New DFlash support in oMLX 0.3.5 RC1 achieves 2x speedup for Qwen3.5 27B inference on Apple Silicon, reaching 22 T/S from 9 T/S using speculative decoding with draft models.
-
Local LLMs on Apple Silicon Mac 2026: M1 M2 M3 Guide
A comprehensive guide from SitePoint covering the latest techniques and models optimized for running local LLMs on Apple Silicon Macs in 2026. Essential reading for macOS users seeking practical deployment strategies.
-
Achieving 2000 Tokens Per Second with QWEN 3.5 27B on RTX-5090
A practitioner shares real-world performance benchmarks achieving 2000 TPS with QWEN 3.5 27B optimized for document classification workloads on consumer-grade RTX-5090 hardware.
-
Mojo: Creating a Programming Language for an AI World with Chris Lattner
A video discussion on Mojo, a programming language designed specifically for AI workloads, offering insights into language design for efficient local model training and inference.
-
Every agent framework has the same bug – prompt decay. Here's a fix
A critical analysis identifies prompt decay as a common vulnerability in agent frameworks, where model outputs gradually degrade over extended interactions. A practical fix is proposed and shared.
-
Scaling llama.cpp On Neoverse N2: Solving Cross-NUMA Performance Issues
Deep dive into optimizing llama.cpp performance on ARM Neoverse N2 processors, addressing critical NUMA topology challenges for better local inference scaling.