Tagged "gpu-inference"
8 articles tagged gpu-inference, 3 March 2026 to 10 October 2026. Newest first.
-
ML Drift: Google's Open-Source On-Device GPU Inference Engine
Google has released ML Drift, a new open-source GPU inference engine specifically optimized for edge and on-device deployment scenarios. The framework focuses on GPU efficiency for local inference workloads.
-
Qwen3.8-27B: Running a Frontier-class Open Model on Your Local GPU
A comprehensive guide to deploying Qwen3.8-27B, a frontier-class open model, on consumer GPUs with practical optimization techniques for local inference.
-
llama.cpp b10549: Tensor Parallelism Support for LFM2/LFM2MOE Models
Latest llama.cpp release enables tensor split for LFM2 and LFM2MOE models, expanding multi-GPU inference capabilities for local deployment.
-
Intel-Scaler-vLLM 0.21.0-b1 Brings Latest Features for vLLM on Intel GPUs
vLLM, the high-performance LLM inference engine, has released version 0.21.0-b1 with optimized support for Intel GPUs. This update enables developers to leverage Intel's discrete graphics for efficient local model serving.
-
Nvidia Enters Windows Laptop Market, Taking on Intel and AMD
Nvidia's entry into the Windows laptop GPU market with dedicated consumer hardware expands the available options for local LLM deployment on consumer machines and edge devices.
-
Prefill Is Compute-Bound, Decode Is Memory-Bound: Optimizing GPU Utilization for LLM Inference
A deep dive into why GPUs shouldn't handle both prefill and decode phases equally, and how understanding this fundamental bottleneck can dramatically improve local LLM inference performance.
-
Researcher Successfully Runs Local LLMs on Legacy "Dead" GPU With Surprising Results
An experiment demonstrates that older or supposedly obsolete GPUs can still effectively run local language models through optimized inference techniques. This discovery makes local LLM deployment accessible to users with older hardware.
-
Intel Arc Pro B70 Workstation GPU Confirmed via vLLM AI Release Notes
Intel's Arc Pro B70 discrete GPU receives official support in vLLM release notes, expanding local LLM inference options for professional workstations. The BMG-G31 architecture targets professional AI computing workflows.