Tagged "sglang"
4 articles tagged sglang, 21 April 2026 to 6 November 2026. Newest first.
-
GPUWorks: 40 MB GPU Cluster Manager for vLLM and SGLang
Minimal-footprint cluster manager enables multi-GPU LLM serving without Kubernetes, reducing operational overhead for self-hosted inference. Demonstrates increasing focus on lightweight orchestration solutions.
-
Google Announces vLLM and SGLang Inference Engines on TPU
Google has released new vLLM and SGLang inference engine optimizations for TPU, enabling faster and more efficient local LLM deployment on Google's tensor processing units with improved performance characteristics.
-
Prefill Concurrency in SGLang: Consistent TTFT Under Multi-Tenant Load
SGLang's new prefill concurrency feature addresses head-of-line blocking in multi-tenant LLM serving, maintaining consistent time-to-first-token even under variable request loads. This improves the viability of shared local LLM deployments.
-
Malicious GGUF Models Could Trigger Remote Code Execution on SGLang Servers
Security researchers have identified a critical vulnerability where specially crafted GGUF model files can achieve remote code execution on SGLang inference servers, posing significant risks to organizations running local LLM deployments.