Tagged "software-optimization"
9 articles tagged software-optimization, 11 February 2026 to 8 September 2026. Newest first.
-
Ollama Replacement 2-4x Faster for No Extra Compute Cost
A new project offers 2-4x faster LLM inference performance compared to Ollama without requiring additional computational resources. This optimization addresses a key pain point for local deployment practitioners seeking faster model serving.
-
Nvidia Boosts Token Throughput 5x With Software Optimizations, Reshaping AI Inference Economics
Nvidia achieves a 5x improvement in token throughput for LLM inference through software optimizations in vLLM, dramatically improving the economics of local and self-hosted model deployment. This breakthrough demonstrates that software efficiency can match or exceed hardware upgrades for inference workloads.
-
Meet EAGLE 3.1: The Speculative Decoding Algorithm That Fixes Attention Drift in LLM Inference
EAGLE 3.1 introduces an improved speculative decoding approach that addresses attention drift, significantly improving inference speed and efficiency for local LLM deployment.
-
ROCm 7.2.3 Delivers Performance Improvements Over 7.0.0 on AMD Radeon AI PRO
Phoronix benchmarks show measurable performance gains with ROCm 7.2.3 compared to version 7.0.0 on AMD's Radeon AI PRO R9700 GPU. The improvements highlight the importance of staying current with driver and runtime updates for optimal local inference performance.
-
One LM Studio Setting Change Makes Local LLMs Competitive With Cloud Models
A simple configuration adjustment in LM Studio dramatically improves local LLM performance, making self-hosted inference viable for production workloads previously requiring cloud APIs. This discovery highlights how software optimization can rival hardware improvements.
-
Researcher Successfully Runs Local LLMs on Legacy "Dead" GPU With Surprising Results
An experiment demonstrates that older or supposedly obsolete GPUs can still effectively run local language models through optimized inference techniques. This discovery makes local LLM deployment accessible to users with older hardware.
-
Build a $1,500 AI Server with DeepSeek-R1 on RTX 4090
Practical guide for assembling and configuring a sub-$1,500 AI inference server using NVIDIA RTX 4090 and DeepSeek-R1, including setup instructions and performance expectations for local deployments.
-
HP ZBook Ultra 14 G1a Workstation Reclaims Local AI Workflows for Professionals
A detailed review of the HP ZBook Ultra 14 G1a demonstrates how modern workstation-class laptops enable practical local AI model deployment for professional workflows. The review evaluates performance and suitability for on-device inference tasks.
-
Developer Switches from Ollama and LM Studio to llama.cpp for Better Performance
A detailed comparison reveals why switching to raw llama.cpp can provide better control and performance for local LLM deployment compared to popular GUI tools.