Tagged "inference-framework"
6 articles tagged inference-framework, 3 March 2026 to 4 September 2026. Newest first.
-
Perplexity Open-Sources Lily: 1.35x Faster Inference Than MLX on Apple Silicon
Perplexity releases Lily, an optimised inference framework for Apple Silicon delivering 1.35x speedup compared to MLX, expanding the tooling ecosystem for on-device LLM inference on M-series Macs.
-
Ollama v0.32.15 Release
Latest Ollama update continues refinement of the popular local LLM inference framework with performance improvements and stability enhancements across platforms.
-
AMD Lemonade Enables Local AI Portability With New Nvidia Support
A practitioner switched their local AI setup to AMD's Lemonade framework after Nvidia support was added, solving key portability challenges. This development demonstrates growing software ecosystem maturity for AMD-based local inference.
-
Can IBM's RITS Platform and vLLM Reset the Bar for Enterprise AI Access?
IBM's RITS platform combined with vLLM is positioning local and on-premises LLM deployment as a viable enterprise alternative, with improved accessibility and control.
-
Malicious GGUF Models Could Trigger Remote Code Execution on SGLang Servers
Security researchers have identified a critical vulnerability where specially crafted GGUF model files can achieve remote code execution on SGLang inference servers, posing significant risks to organizations running local LLM deployments.
-
Framework Choice Critical: llama.cpp and vLLM Outperform Ollama for Qwen 3.5 Testing
Community PSA reveals significant performance and correctness differences between local inference frameworks when running Qwen 3.5 models, with llama.cpp, transformers, vLLM, and SGLang producing correct results while Ollama shows issues with reasoning and tool use.