Tagged "multi-gpu"
6 articles tagged multi-gpu, 15 March 2026 to 15 November 2026. Newest first.
-
NVIDIA Tutorial: Serving GLM-5.3 at 607 Tokens/Second Across Two DGX Stations with vLLM
NVIDIA publishes guidance for serving the GLM-5.3 model across distributed DGX hardware using vLLM, achieving 607 tokens/second throughput and demonstrating multi-GPU orchestration for demanding inference workloads.
-
Llama.cpp Fork Achieves 2-4x MultiGPU Speedup for MoE Models Larger Than VRAM
A community fork of llama.cpp enables efficient distributed inference for Mixture-of-Experts models that exceed single GPU VRAM capacity, achieving 2-4x speedup improvements across multiple GPUs.
-
Titan Transients and LLM Scalability
An ACM Queue article examining scalability challenges and solutions for large language models, relevant to understanding infrastructure requirements for local deployment scenarios.
-
Ray Serve LLM Achieves 24x Performance Improvement in Distributed Inference
Ray Serve LLM has demonstrated significant performance enhancements in distributed inference scenarios, delivering up to 24x faster throughput for locally-hosted model serving.
-
Most People Use Ollama or llama.cpp for Local LLMs, but These Are the Tools I Switch to When It Gets Serious
An experienced practitioner compares advanced local LLM deployment tools beyond the popular Ollama and llama.cpp, highlighting specialized frameworks for production scenarios.
-
Running Qwen3.5-27B Across Multiple GPUs Over LAN Achieves Practical Speed for Local Inference
A practitioner successfully split Qwen3.5-27B across a 4070Ti and AMD RX6800 over LAN using llama.cpp's RPC server, achieving 13 tokens/second with 32K context—demonstrating that heterogeneous multi-GPU local setups are now viable. This shows path forward for GPU-poor practitioners seeking reasonable performance.