Tagged "multi-gpu-orchestration"
3 articles tagged multi-gpu-orchestration, 15 March 2026 to 15 November 2026. Newest first.
-
NVIDIA Tutorial: Serving GLM-5.3 at 607 Tokens/Second Across Two DGX Stations with vLLM
NVIDIA publishes guidance for serving the GLM-5.3 model across distributed DGX hardware using vLLM, achieving 607 tokens/second throughput and demonstrating multi-GPU orchestration for demanding inference workloads.
-
Netflix Details Its In-House LLM Serving Platform with Triton and vLLM
Netflix has published details about its production LLM serving infrastructure, combining NVIDIA Triton and vLLM for efficient model deployment. This real-world case study demonstrates battle-tested patterns for scaling LLM inference at enterprise scale.
-
Running Qwen3.5-27B Across Multiple GPUs Over LAN Achieves Practical Speed for Local Inference
A practitioner successfully split Qwen3.5-27B across a 4070Ti and AMD RX6800 over LAN using llama.cpp's RPC server, achieving 13 tokens/second with 32K context—demonstrating that heterogeneous multi-GPU local setups are now viable. This shows path forward for GPU-poor practitioners seeking reasonable performance.