Tagged "inference-architecture"

3 articles tagged inference-architecture, 26 February 2026 to 5 May 2026. Newest first.

• Supercharging LLM Inference on Google TPUs: Achieving 3X Speedups With Diffusion-Style Speculative Decoding 5 May 2026

  Google researchers have demonstrated 3x inference speedups on TPUs using diffusion-style speculative decoding, a novel optimization technique that could influence local inference strategies. The breakthrough shows how advanced decoding methods can dramatically reduce latency on specialized hardware.

• Prefill Is Compute-Bound, Decode Is Memory-Bound: Optimizing GPU Utilization for LLM Inference 16 April 2026

  A deep dive into why GPUs shouldn't handle both prefill and decode phases equally, and how understanding this fundamental bottleneck can dramatically improve local LLM inference performance.

• DeepSeek Paper – DualPath: Breaking the Bandwidth Bottleneck in LLM Inference 26 February 2026

  DeepSeek researchers present DualPath, a novel approach to address bandwidth limitations during LLM inference. This work tackles one of the primary performance bottlenecks in local and edge LLM deployment.