FreeToken: Edge-Native MoE Serving Engine for Consumer Hardware

24 August 2026 1 min read

#consumer-gpu #deepseek #glm #memory-optimization #mixture-of-experts #moe #quantization #release #zhipu

FreeToken is a mixture-of-experts (MoE) serving engine built for edge-native inference — running frontier-scale open-weight models on personal and consumer hardware rather than on cluster infrastructure. Its stated target is 290B+ parameter MoE models on a gaming PC, and the supported list includes GLM-5.2, DeepSeek-V4-Flash and Qwen3.6-35B-A3B.

The approach is not single-GPU. FreeToken treats heterogeneous edge resources — GPUs, CPUs, host memory and interconnects — as one elastic pool, with bandwidth-adaptive CPU–GPU co-execution, expert caching, and dynamic reallocation of VRAM between expert caches and KV memory at runtime.

The project publishes a paper (arXiv 2608.16157) describing the bandwidth-adaptive execution scheme.

Read the full article on Google News.

Source: Google News · Relevance: 9/10