Tagged "batched-inference"
3 articles tagged batched-inference, 11 February 2026 to 27 August 2026. Newest first.
-
vLLM v0.28.0 Features Major Kimi-K3 Optimization and Decode Context Parallel Support
vLLM 0.28.0 introduces Decode Context Parallel (DCP) support and optimized kernels for Kimi-K3, alongside improvements for 270+ contributors. The release enables faster multi-sequence inference on both datacenter and edge hardware.
-
NVIDIA Accelerates Gemma 4 for Local Agentic AI on RTX GPUs
NVIDIA provides day-one optimizations for Google's Gemma 4 models across its RTX GPU lineup, enabling accelerated local inference for agentic AI workflows on consumer and enterprise graphics cards.
-
Mistral AI Debugs Critical Memory Leak in vLLM Inference Engine
Mistral AI's engineering team shares their process for identifying and fixing a significant memory leak in vLLM that was affecting production deployments.