Tagged "moe-inference"

3 articles tagged moe-inference, 26 February 2026 to 1 September 2026. Newest first.

• FreeToken: Edge-Native MoE Serving with CPU-GPU Co-Execution 1 September 2026

  FreeToken is an open-source engine for running 290B+ Mixture-of-Experts models locally on consumer hardware through bandwidth-adaptive CPU-GPU co-execution, with elastic memory management, expert caching, and support for DeepSeek, Qwen and GLM models across NVIDIA RTX 30/40/50 series.

• FreeToken: Edge-Native MoE Serving Engine for Consumer Hardware 24 August 2026

  FreeToken is a mixture-of-experts serving engine aimed at running frontier-scale open-weight models on consumer hardware, using CPU-GPU co-execution rather than a multi-GPU cluster.

• Qwen 3.5 MoE Delivers 100K Context Window at 40+ TPS on RTX 5060 Ti 26 February 2026

  Qwen3.5's mixture-of-experts variant achieves exceptional throughput with 100,000 token context window on a single mid-range GPU, reaching 41+ tokens per second using the Vulkan backend. This demonstrates practical feasibility of ultra-long context models on consumer hardware.