Ollama 0.40.17 Improves Vulkan Integrated GPU and mmap Support

1 min read
GitHubpublisher

Ollama 0.40.17 delivers practical improvements for running models on integrated GPUs via Vulkan. The addition of mmap (memory-mapped file I/O) support for Vulkan integrated GPUs is particularly significant for laptop and desktop users who lack discrete graphics but have modern integrated graphics that can accelerate inference. This optimization reduces memory pressure and improves access patterns when moving model weights into GPU memory.

The release also stabilizes CLI restore mechanisms and graceful restart handling, addressing reliability concerns for production deployments and long-running inference services. For practitioners managing Ollama instances in production or demanding high availability, these improvements reduce operational incidents around model reloading and service restart cycles.

Read the full article on Ollama release.


Source: Ollama release · Relevance: 9/10