Ollama v0.32.6: Faster Apple GPU Inference with Speculative Decoding

Ollama v0.32.6 brings meaningful performance optimizations for local LLM deployment on Apple devices. The MLX engine now automatically leverages model MTP heads for speculative decoding on Qwen3.5 and compatible models, enabling faster token generation without requiring manual configuration.

Additionally, the release improves streaming compatibility by matching OpenAI's wire format exactly—role information only appears on the first chunk and finish_reason on its own chunk. This makes Ollama a drop-in replacement for existing applications expecting OpenAI-compatible endpoints, reducing friction for developers migrating to local inference.

For practitioners running inference on MacBooks and Mac Studios, these improvements directly translate to faster response times and better resource utilization, making local deployment more practical for production workloads.

Read the full article on Ollama release.

Source: Ollama release · Relevance: 9/10