Tagged "hardware-utilization"
7 articles tagged hardware-utilization, 23 February 2026 to 1 July 2026. Newest first.
-
Asahi Linux 7.1 Progress Report
Latest progress on Asahi Linux, Apple Silicon's open-source Linux distribution, which is critical infrastructure for local LLM deployment on Mac hardware. Updates include improved hardware utilisation and performance optimisations.
-
vLLM vs Ollama 2026: Performance Benchmark Reveals 9x Throughput Gap
A comprehensive benchmark comparison shows vLLM significantly outperforming Ollama in throughput metrics, with implications for choosing the right inference framework for local deployments.
-
llama.cpp Adds Multi-Token Prediction, Doubles Qwen 3.6B Throughput for Local Inference
llama.cpp, the popular C++ inference engine for local LLMs, has added multi-token prediction capabilities and achieved a 2x throughput improvement on Qwen 3.6B models. This breakthrough enables faster token generation for on-device deployments without sacrificing accuracy.
-
Elastic KV Cache Memory Breakthrough Enables Efficient Bursty LLM Serving and GPU Sharing
A new coding implementation on elastic KV cache memory optimization allows more efficient handling of variable-load LLM serving patterns and multi-model GPU sharing scenarios.
-
GPU Passthrough to LXCs in Proxmox Simplifies Local Inference Infrastructure
System administrators discover that GPU passthrough to Linux containers in Proxmox offers simpler and more efficient deployment for local LLM inference compared to traditional virtual machines. This reduces operational complexity for self-hosted inference setups.
-
DGX Spark Hardware Limitations: Missing NVFP4 Support Undermines Local AI Value Proposition
User experience reports reveal that NVIDIA's DGX Spark lacks critical NVFP4 (NV Tensor Float 32) support six months after launch, significantly limiting its utility for cost-effective local model inference despite Blackwell GPU capabilities.
-
Yet Another Fix Coming for Older AMD GPUs on Linux – Thanks to Valve Developer
Valve developers continue improving AMD GPU support on Linux, bringing better hardware compatibility for local LLM inference. This ongoing effort makes older AMD hardware more viable for local model deployment.