Tagged "inference-latency-reduction"
3 articles tagged inference-latency-reduction, 1 March 2026 to 21 April 2026. Newest first.
-
Gemma 4 Just Replaced My Whole Local LLM Stack
Google's Gemma 4 model is making waves in the local LLM community as developers report it outperforms their existing local inference setups. The model appears to offer significant improvements in capability-to-size ratio, making it an attractive option for on-device deployment.
-
Strix Halo (Ryzen AI Max+ 395) Achieves Strong Local Inference Performance with ROCm 7.2
New benchmarks on AMD's Strix Halo platform with ROCm 7.2 backend show practical inference speeds for the Qwen 3.5 model family, with recent llama.cpp optimisations delivering measurable performance gains.
-
Switch Qwen 3.5 Thinking Mode On/Off Without Model Reload Using setParamsByID
Unsloth and Qwen community members have discovered how to toggle thinking vs. instruct mode on Qwen 3.5 without reloading the model, enabling dynamic workflow switching and reducing inference latency.