Tagged "llama-cpp-optimization"
13 articles tagged llama-cpp-optimization, 11 February 2026 to 26 May 2026. Newest first.
-
Developer Switches from LM Studio to llama.cpp, Reports No Performance Downgrade
A developer shares their experience migrating from LM Studio to llama.cpp for local LLM inference, finding the lighter-weight tool delivers comparable performance with better resource efficiency.
-
llama.cpp Checkpoint Fix Accelerates Local Coding Agents
An optimization to llama.cpp's checkpoint handling improves inference speed for coding agent tasks, delivering faster token generation for local development workflows.
-
Dynamic Expert Cache in llama.cpp Achieves 27% Faster Inference on Large MoE Models
A new optimization technique for llama.cpp improves CPU+GPU token generation speed by 27% on Qwen3.5-122B through dynamic expert caching, raising practical inference rates from 15 to 23 tokens per second.
-
Users Report Significant Performance Improvements After Migrating from Ollama to llama.cpp
Local LLM practitioners are experiencing notable speed and stability improvements when switching from Ollama to direct llama.cpp implementations, suggesting framework-level optimization differences in inference throughput and reliability.
-
Gemini-CLI, Llama.cpp, and Qwen3.5 Running on NVIDIA Jetson TK1
Community members report successfully running multiple LLMs including Qwen3.5 and Gemini models via llama.cpp on NVIDIA Jetson TK1 edge devices, showcasing practical deployment on resource-constrained embedded hardware.
-
TurboQuant-Optimized llama.cpp Fork Delivers GFX906 GPU Acceleration
Community developer releases optimized llama.cpp fork featuring TurboQuant quantization and specialized GFX906 GPU optimizations with Gemma 4 architecture support coming soon.
-
SmolLM2-360M Running on Samsung Galaxy Watch 4 with 74% Memory Reduction
Developer optimizes llama.cpp to run language models on smartwatches, achieving 74% RAM reduction through memory model improvements and reducing peak usage from 524MB to practical levels.
-
ik_llama.cpp Fork Delivers 26x Faster Prompt Processing on Qwen 3.5 27B
A fork of llama.cpp called ik_llama.cpp is delivering dramatic 26x speed improvements for prompt processing on Qwen 3.5 27B models. Real-world benchmarks on Blackwell RTX PRO GPUs show tangible performance gains for production agentic workloads.
-
Practical Fix for Qwen 3.5 Overthinking in llama.cpp
Community members share techniques to mitigate Qwen 3.5's verbose internal reasoning loops, offering practical optimization strategies for controlling model behavior in local inference environments.
-
Strix Halo (Ryzen AI Max+ 395) Achieves Strong Local Inference Performance with ROCm 7.2
New benchmarks on AMD's Strix Halo platform with ROCm 7.2 backend show practical inference speeds for the Qwen 3.5 model family, with recent llama.cpp optimisations delivering measurable performance gains.
-
Llama.cpp Prompt Processing Optimization: Ubatch Size Configuration Guide
A community member shares practical troubleshooting advice for improving prompt processing performance on larger models like Qwen 27B by configuring ubatch size parameters in llama.cpp.
-
Scaling llama.cpp On Neoverse N2: Solving Cross-NUMA Performance Issues
Deep dive into optimizing llama.cpp performance on ARM Neoverse N2 processors, addressing critical NUMA topology challenges for better local inference scaling.
-
Developer Switches from Ollama and LM Studio to llama.cpp for Better Performance
A detailed comparison reveals why switching to raw llama.cpp can provide better control and performance for local LLM deployment compared to popular GUI tools.