Tagged "llama-cpp"
-
Picking Your First Local LLM Is Easier Than the Internet Makes It Sound
-
NVIDIA Nemotron 3 Nano Omni Powers Multimodal Agent Reasoning in a Single Efficient Open Model
-
Llama.cpp Runs on SGI Power Challenge from 1995 with MIPS R8000 Kernel
-
Grokfeed: Terminal Feed Reader for HN, Reddit, and Lobste.rs Using Claude Code
-
Local AI Isn't Just Ollama—Here's the Ecosystem That Actually Makes It Useful
-
Hipfire: A Rust-Native AMD Inference Engine That Outperforms llama.cpp
-
An Update on GitHub Availability: Infrastructure Lessons for Hosted LLM Tools
-
Linux Crushes Windows on llama.cpp Inference by Double Digits
-
Run a Local LLM Server on Raspberry Pi with Remote Access Capabilities
-
I Replaced My Local LLM With a Model Half Its Size and Got Better Results
-
Building Real-World On-Device AI with LiteRT and NPU
-
Intel OpenVINO 2026.1 Integrates llama.cpp with Wildcat Lake and Arc Pro B70
-
Llama.cpp's Auto Fit Feature Quietly Reshapes Local AI Inference on Consumer Hardware
-
The Open-Source AI Ecosystem Keeps Treating llama.cpp Like a Second-Class Citizen
-
Malicious GGUF Models Could Trigger Remote Code Execution on SGLang Servers
-
llama.cpp Merges Speculative Checkpointing for Major Inference Speed Boost
-
Bun v1.3.13
-
AI Quota Inflation Is No Token Effort. It's Baked In
-
Local AI Isn't Just Ollama—Here's the Ecosystem That Actually Makes It Useful
-
LlaMa.cpp Robot Wars
-
Kilo is the VS Code Extension That Actually Works with Every Local LLM
-
Unweight: Lossless MLP Weight Compression for LLM Inference
-
Sorting 1M u64 KV-Pairs in 20ms on i9-13980HX Using Branchless Rust Implementation
-
Kilo Is the VS Code Extension That Actually Works With Every Local LLM I Throw at It
-
The 'Ollama' Tool Has Numerous Problems, and Some Argue That Llama.cpp Is Better
-
ChatMCP – Connect your AI browser chats to your coding agents
-
Project Glasswing and the ASF: Open-Source's Chance to Win the AI Era
-
Dynamic Expert Cache in llama.cpp Achieves 27% Faster Inference on Large MoE Models
-
DotLLM – Building an LLM Inference Engine in C#
-
Sovereign AI: Why the Next GPT Will Be Born in Our Living Rooms
-
Qwen 3.5 Small – On-Device Multimodal Models Released
-
Speculative Decoding Achieves 29% Speed Boost for Gemma-4 31B
-
Self-Hosted LLM Took Personal Knowledge Management System to the Next Level
-
Qwen3 Audio and Vision Support Now Available in llama.cpp
-
MiniMax M2.7 Open-Sources Globally as Industry's First Self-Improving Model
-
Audio Processing Support Lands in llama.cpp with Gemma-4
-
ASUS Malaysia to Bring UGen300 USB AI Accelerator in Q2 for Portable On-Device AI Inferencing
-
Users Report Significant Performance Improvements After Migrating from Ollama to llama.cpp
-
MiniMax M2.7 Is Now Open Source
-
Critical Unsloth Gemma-4 Chat Template Updates for Tool Calling
-
Intel Arc Pro B70 32GB Achieves 12 Tokens/Sec on Qwen 3.5-27B
-
Tether Launches QVAC SDK for Cross-Platform Local AI Development
-
Ollama's Limitations for Production Local LLM Deployments
-
Gemma 4 Template Improvements Enhance Tool Use and Dialog Compliance
-
Speculative Decoding Made My Local LLM Actually Usable
-
Ollama is Still the Easiest Way to Start Local LLMs, But It's the Worst Way to Keep Running Them
-
Gemini-CLI, Llama.cpp, and Qwen3.5 Running on NVIDIA Jetson TK1
-
Intel Releases OpenVINO 2026.1 With Backend For Llama.cpp, New Hardware Support
-
Gemma 4 Support Stabilized in Llama.cpp
-
Gemma 4 GGUF Models Updated with Critical Quantization Fixes
-
EXAONE 4.5 33B Model Released with Multiple Quantization Formats
-
MemPalace, the Highest-Scoring AI Memory System Ever Benchmarked
-
TurboQuant-Optimized llama.cpp Fork Delivers GFX906 GPU Acceleration
-
TurboQuant in Llama.cpp Achieves 6X Smaller KV Cache
-
Quantization Strategy Comparison: Balancing Quality and Speed on Consumer Laptops
-
Context Window Optimization: Extending Gemma 4 Context Length Through Efficient Projection Quantization
-
GPU Memory for LLM Inference (Part 1)
-
Google AI Edge Gallery Tops App Store Charts with On-Device Gemma 4
-
Vektor – Local-First Associative Memory for AI Agents
-
Unpaved: Audit Toolkit for AI Developer Tool Bias in Global South Contexts
-
Qwen 3.6 Free Model Available via OpenRouter
-
Microsoft Quantum Development Kit Ported to Rust: 100x Faster and Smaller
-
Apple Research Shows Self-Distillation Significantly Improves Local Code Generation
-
GPUs vs. TPUs: Decoding the Powerhouses of AI
-
Gemma 4 KV Cache Memory Issues Fixed in llama.cpp
-
OpenUMA – Apple-Style Unified Memory for x86 AI Inference
-
VRAM Optimization Technique Cuts Gemma 4 Memory Usage by 3x
-
Google Gemma 4 Released with GGUF Quantizations
-
Gemma 4 2B Successfully Runs on Raspberry Pi 5
-
SmolLM2-360M Running on Samsung Galaxy Watch 4 with 74% Memory Reduction
-
Intel's $949 GPU Has 32GB of VRAM for Local AI, but Software is Why Nvidia Keeps Winning
-
Show HN: Extra-Platforms, Python Library to Detect OS, Arch, Shell, CI, AI
-
ROCm Integration in Ubuntu 26.04 Advances Linux GPU Inference
-
Local AI Ecosystem Extends Far Beyond Ollama
-
Llama.cpp Merging TurboQuant Lite (attn-rot) with Major Performance Gains
-
Gemini CLI – Open-Source AI Agent for Terminal Integration
-
Samsung launches Galaxy Book6 series in India with Nvidia RTX 5070 graphics and on-device AI
-
Intel's $949 GPU has 32GB of VRAM for local AI, but the software is why Nvidia keeps winning
-
Closed Source AI = Neofeudalism
-
DeepSeek V3 Complete Guide: Deploy and Optimize Local AI in 2026
-
Local AI Ecosystem Extends Far Beyond Ollama
-
Unsloth Studio Beta Ships 50+ New Features for Local Model Training and Inference
-
TurboQuant KV Cache Compression Achieves 22.8% Faster Decoding at 32K Context
-
Introduction to Nyreth v1.0
-
HP Launches Copilot+ PCs in India with On-Device AI Capabilities for Local Inference
-
TurboQuant Benchmarked in Llama.cpp: Google's Extreme Compression Research Tested in Practice
-
RotorQuant: 10-19x Faster Quantisation Alternative Using Clifford Algebra
-
Coding Implementation to Run Qwen3.5 Reasoning Models Distilled With Claude-Style Thinking Using GGUF and 4-Bit Quantization
-
Quantization Reveals Outliers Impacting LLM Accuracy
-
Pluggable's TBT5-AI: First Thunderbolt Dock Explicitly Targeting Local LLM Workstations
-
Nota AI and SiMa.ai Partner on Physical AI Technology for Local Deployment
-
Google's TurboQuant: The Unsexy AI Breakthrough Worth Watching
-
Apple Plans Slimmed-Down Gemini Models for Local iPhone AI Features
-
Show HN: Open Agent Spec – Treat AI Agents Like Typed Functions, Not Prompt Chains
-
OmniCoder v2 Released: Improved Code Generation for Local Deployment
-
Researcher Successfully Runs Local LLMs on Legacy "Dead" GPU With Surprising Results
-
Llama.cpp Benchmark: RTX 5090 vs Enterprise Systems Compared
-
I built Rubric, an open source Sentry for AI. Looking for beta testers
-
LM Studio Releases Reworked Plugins with Fully Local Web Research
-
Llama.cpp ROCm 7 vs Vulkan Performance Benchmarks on AMD Mi50
-
Rust Project Perspectives on AI
-
Setting Up a Private AI Brain on Windows: Complete Guide to Local LLM Deployment
-
ik_llama.cpp Fork Delivers 26x Faster Prompt Processing on Qwen 3.5 27B
-
Careless Whisper – Personal Local Speech to Text
-
Automating Read-It-Later Workflows with Local LLMs for Overnight Summarization
-
Qualcomm and Samsung's 30-Year AI Alliance Enters a New Phase as On-Device AI Chip Race Heats Up
-
What AI Augmentation Means for Technical Leaders
-
Community Converges on Optimal KV Cache Quantization Strategies for Qwen 3.5 Models
-
LMCache Dramatically Accelerates LLM Inference on Oracle Data Science Platform
-
Kilo Is the VS Code Extension That Actually Works With Every Local LLM I Throw At It
-
Unsloth Studio: Open-Source Web UI for Training and Running LLMs Locally
-
On-Device AI: Tether's QVAC Fabric Enables Local Training
-
I Switched to a Local LLM for These 5 Tasks and the Cloud Version Hasn't Been Worth It Since
-
LucidShark – Local-first, open-source quality and security gate
-
You're Using Your Local LLM Wrong If You're Prompting It Like a Cloud LLM
-
Hugging Face Releases One-Liner for Automatic Hardware Detection and Model Selection
-
Run LLMs Locally with Llama.cpp
-
I Ran Local LLMs on a 'Dead' GPU, and the Results Surprised Me
-
Mistral Releases Small 4 Open-Source Model Under Apache 2.0
-
How I Used Lima for an AI Coding Agent Sandbox
-
Kimi Introduces Attention Residuals: 1.25x Compute Performance at <2% Overhead
-
Practical Fix for Qwen 3.5 Overthinking in llama.cpp
-
This External GPU Enclosure Tries to Break Cloud Dependence for Local AI Inference
-
Apple's On-Device AI Raises Privacy Alarms Across British Parliament
-
AMD Declares 'AI on the PC Has Crossed an Important Line' – Agent Computers as Next Breakthrough
-
OpenClaw vs Eigent vs Claude Cowork: Comparing Open-Source AI Collaboration Platforms
-
Running Qwen3.5-27B Across Multiple GPUs Over LAN Achieves Practical Speed for Local Inference
-
AMD Launches Agent System Optimized for Local AI Inference With Ryzen and Radeon
-
Intel OpenVINO Backend Support Now Available in llama.cpp
-
Memory Should Decay: Implementing Temporal Memory Decay in Local LLM Systems
-
How to Run Local LLMs in 2026: The Complete Developer's Guide
-
AgentArmor: Open-Source 8-Layer Security Framework for AI Agents
-
3-Path Agent Memory: 8 KB Recurrent State vs. 156 MB KV Cache at 10K Tokens
-
Quantization Explained: Q4_K_M vs AWQ vs FP16 for Local LLMs
-
Nvidia Releases Nemotron 3 Super: 120B MoE Model for Local Deployment
-
Llama.cpp Adds True Reasoning Budget Support
-
Cutile.jl Brings Nvidia CUDA Tile-Based Programming to Julia
-
SK Hynix Completes Qualification for LPDDR6 Memory Optimized for AI Inference
-
Sarvam Open-Sources 30B and 105B Reasoning Models
-
NVIDIA Jetson Brings Open Models to Life at the Edge
-
LMF – LLM Markup Format
-
Llama.cpp Celebrates Major Milestone: From Leak to Industry Standard
-
Qwen 3.5 Ultra-Compact Models Enable On-Device AI from Watches to Gaming
-
Mnemos: Persistent Memory System for Local AI Agents
-
FreeBSD 14.4 Released: Implications for Local LLM Deployment
-
M5 Max and M5 Ultra Chipsets Demonstrate Significant Bandwidth Improvements for Local LLM Inference
-
Community Survey: AI Content Automation Stacks in 2026
-
Strix Halo (Ryzen AI Max+ 395) Achieves Strong Local Inference Performance with ROCm 7.2
-
Sarvam Open-Sources 30B and 105B Reasoning Models
-
Llama.cpp Prompt Processing Optimization: Ubatch Size Configuration Guide
-
HP Refreshes Lineup with AI-Focused Workstations
-
Llama.cpp Merges Automatic Parser Generator to Mainline
-
Turning Your Linux Terminal into a Local AI Assistant
-
llama.cpp Merges Agentic Loop and MCP Client Support
-
Apple Unveils MacBook Pro with M5 Pro and M5 Max Featuring On-Device AI
-
OpenWrt 25.12.0 – Stable Release
-
Apple Unveils MacBook Pro With M5 Pro and M5 Max for On-Device AI
-
AMD Launches Copilot+ Desktop Chips to Compete in On-Device AI Market
-
ÆTHERYA Core – Deterministic Policy Engine for Governing LLM Actions
-
Qwen 3.5 Small Models Released: 0.8B to 9B Parameters Optimized for On-Device Inference
-
Qwen 3.5 0.8B Successfully Deployed on 7-Year-Old Samsung S10E Using llama.cpp
-
Framework Choice Critical: llama.cpp and vLLM Outperform Ollama for Qwen 3.5 Testing
-
Critical: Qwen 3.5 Requires BF16 KV Cache, Not FP16 for Accurate Inference
-
GitDelivr: A Free CDN for Git Clones Built on Cloudflare Workers and R2
-
C7: Pipe Up-to-Date Library Docs Into Any LLM From the Terminal
-
Huawei's SuperPoD Portfolio Creates New Option for Global Computing at MWC Barcelona 2026
-
Unsloth Dynamic 2.0 GGUFs
-
5 Useful Docker Containers for Agentic Developers
-
Seco Launches Edge AI System-on-Module at Embedded World 2026
-
Arduino and Qualcomm Bring On-Device AI Learning to Indian Schools
-
DeepSeek Releases DualPath: Addressing Storage Bandwidth Bottlenecks in Agentic Inference
-
DeepSeek Paper – DualPath: Breaking the Bandwidth Bottleneck in LLM Inference
-
Qwen3.5 Thinking Mode Can Be Disabled for Production Inference Optimization
-
Qwen3.5-27B Identified as Sweet Spot for Mid-Range Local Deployment
-
Mirai Announces $10M to Advance On-Device AI Performance for Consumer Devices
-
Advanced Quantization Techniques Show Surprising Performance Gains Over Standard Methods
-
How AI is Redefining Price and Performance in Modern Laptops
-
Show HN: A Ground Up TLS 1.3 Client Written in C
-
Enterprise Infrastructure Guide: Running Local LLMs for 70-150 Developers
-
Apple Accelerates U.S. Manufacturing with Mac Mini Production
-
nanollama: Open-Source Framework for Training Llama 3 from Scratch with One-Command GGUF Export
-
Open-Source llama.cpp Finds Long-Term Home at Hugging Face
-
Breaking the Speed Limit: Strategies for 17k Tokens/Sec Local Inference
-
Ouro 2.6B Thinking Model GGUFs Released with Q8_0 and Q4_K_M Quantization
-
Strix Halo Performance Benchmarks: Minimax M2.5, Step 3.5 Flash, Qwen3 Coder
-
I Thought I Needed a GPU to Run AI Until I Learned About These Models
-
Open-Source + AI: ggml Joins Hugging Face, llama.cpp Stays Open—Local AI's Long-Term Home
-
GGML.AI Acquired by Hugging Face
-
PaddleOCR-VL Now Integrated into llama.cpp for Multilingual OCR
-
Kitten TTS V0.8 Released: New State-of-the-Art Super-Tiny TTS Model Under 25 MB
-
Enhanced Quantization Visualization Methods for Understanding LLM Compression Trade-offs
-
Kitten TTS V0.8 Released: State-of-the-Art Super-Tiny Text-to-Speech Model Under 25MB
-
Self-Hosted AI: A Complete Roadmap for Beginners
-
Meet Sarvam Edge: India's AI Model That Runs on Phones and Laptops With No Internet
-
Qwen 3.5-397B-A17B Now Available for Local Inference with Aggressive Quantisation
-
Open-Source Models Now Comprise 4 of Top 5 Most-Used Endpoints on OpenRouter
-
Switching From Ollama And LM Studio To llama.cpp: A Performance Comparison
-
SnowBall Technique Addresses Context Window Limitations in Local LLMs
-
Scaling llama.cpp On Neoverse N2: Solving Cross-NUMA Performance Issues
-
MiniMax Releases M2.5 Model with SOTA Coding and Agent Capabilities
-
MiniMax-M2.5 230B MoE Model Released with GGUF Support for Local Deployment
-
GNOME's AI Assistant Newelle Adds llama.cpp Support and Command Execution
-
Context Management Identified as Real Bottleneck in AI-Assisted Coding
-
Switching From Ollama and LM Studio to llama.cpp: Performance Benefits
-
Optimal llama.cpp Settings Found for Qwen3 Coder Next Loop Issues
-
GitHub Announces Support for Open Source AI Project Maintainers
-
Scaling llama.cpp On Neoverse N2: Solving Cross-NUMA Performance Issues
-
Scaling llama.cpp On Neoverse N2: Solving Cross-NUMA Performance Issues
-
New Header-Only C++ Benchmark Tool for Predictive Models on Raw Binary Streams
-
Developer Switches from Ollama and LM Studio to llama.cpp for Better Performance