Tagged "multi-gpu"

5 articles tagged multi-gpu, 15 March 2026 to 27 September 2026. Newest first.

• Llama.cpp Fork Achieves 2-4x MultiGPU Speedup for MoE Models Larger Than VRAM 27 September 2026

  A community fork of llama.cpp enables efficient distributed inference for Mixture-of-Experts models that exceed single GPU VRAM capacity, achieving 2-4x speedup improvements across multiple GPUs.

• Titan Transients and LLM Scalability 29 July 2026

  An ACM Queue article examining scalability challenges and solutions for large language models, relevant to understanding infrastructure requirements for local deployment scenarios.

• Ray Serve LLM Achieves 24x Performance Improvement in Distributed Inference 19 June 2026

  Ray Serve LLM has demonstrated significant performance enhancements in distributed inference scenarios, delivering up to 24x faster throughput for locally-hosted model serving.

• Most People Use Ollama or llama.cpp for Local LLMs, but These Are the Tools I Switch to When It Gets Serious 15 June 2026

  An experienced practitioner compares advanced local LLM deployment tools beyond the popular Ollama and llama.cpp, highlighting specialized frameworks for production scenarios.

• Running Qwen3.5-27B Across Multiple GPUs Over LAN Achieves Practical Speed for Local Inference 15 March 2026

  A practitioner successfully split Qwen3.5-27B across a 4070Ti and AMD RX6800 over LAN using llama.cpp's RPC server, achieving 13 tokens/second with 32K context—demonstrating that heterogeneous multi-GPU local setups are now viable. This shows path forward for GPU-poor practitioners seeking reasonable performance.