Tagged "inference-runtime"
4 articles tagged inference-runtime, 20 April 2026 to 23 July 2026. Newest first.
-
How To Build Your Own LLM Runtime From Scratch
A comprehensive guide on constructing custom LLM inference runtimes, providing practitioners with deep knowledge to optimize and control local model deployment without relying on black-box frameworks.
-
llama.cpp MTP Leak Fix Stabilizes Local AI Agents
A critical memory leak fix in llama.cpp improves stability for running local AI agents, addressing a significant issue that affected long-running inference workloads.
-
User Migration from LM Studio/Ollama to llama.cpp Shows Growing Preference
Community feedback indicates llama.cpp is becoming the preferred inference runtime for local deployment, driven by superior performance and flexibility compared to GUI-focused alternatives.
-
llama.cpp Merges Speculative Checkpointing for Major Inference Speed Boost
llama.cpp integrates speculative checkpointing techniques to significantly accelerate local AI inference performance, enabling faster token generation on consumer hardware.