Tagged "performance-optimization"
-
Phison and Intel Roll Out aiDAPTIV to Boost Local AI on Intel AI PC Platforms
-
NVIDIA Launches N1X/N1 CPU-GPU SoC for PC Market, Targeting Heavy On-Device AI Users
-
Apple Doubles Down on On-Device AI at WWDC 2026, Setting Privacy-First Strategy
-
vLLM vs Ollama 2026: Performance Benchmark Reveals 9x Throughput Gap
-
Users Report Superior Performance Switching from LM Studio to llama.cpp
-
AMD Unveils Ryzen AI Halo Developer Platform for On-Device AI Workloads
-
User Migration from LM Studio/Ollama to llama.cpp Shows Growing Preference
-
llama.cpp MTP Leak Fix Stabilizes Local AI Agents
-
AI Token Streaming Isn't About SSE vs. WebSockets
-
Bito's AI Architect Improves Claude Opus Task Success Rate by 35%
-
My Thoughts on AI, Part 1: Fears, Opinions, and Mental Journey
-
How to Train Your GPT: Comprehensive Commented Training Guide
-
llama.cpp Delivers Sharp Performance Gains for AMD RDNA3 Users
-
What If AI Systems Weren't Chatbots?
-
One LM Studio Setting Makes Local LLMs Competitive With Cloud Models
-
Sarvam Edge: Indian-Built AI Models Run Offline on Phones and Laptops Without Internet
-
5 Things I Wish Someone Had Told Me Before I Tried Self-Hosting a Local LLM
-
A 49-Line Physics Classifier That Beats kNN on 76% of Benchmarks
-
PFlash Claims 10x Prefill Speedup Over llama.cpp
-
Linux Crushes Windows on llama.cpp Inference by Double Digits
-
Rust Open-Source Headless Browser for AI Agents and Web Scraping
-
GPU Passthrough to LXCs in Proxmox Outperforms VMs and Simplifies Local AI Infrastructure
-
The Open-Source AI Ecosystem Keeps Treating llama.cpp Like a Second-Class Citizen
-
The AI-Ready Product Data Framework for B2B Commerce
-
Sorting 1M u64 KV-Pairs in 20ms on i9-13980HX Using Branchless Rust Implementation
-
The 'Ollama' Tool Has Numerous Problems, and Some Argue That Llama.cpp Is Better
-
Prefill Is Compute-Bound, Decode Is Memory-Bound: Optimizing GPU Utilization for LLM Inference
-
The Best Local AI Model for Home Assistant Isn't Always the Biggest One
-
Warp Decode vs. vLLM's Triton Kernel: Performance Crossover Analysis
-
Gemma 4 Template Improvements Enhance Tool Use and Dialog Compliance
-
Speculative Decoding Made My Local LLM Actually Usable
-
I Replaced My Local LLM With a Model Half Its Size and Got Better Results — and It Wasn't About the Parameters
-
Gemma 4 GGUF Models Updated with Critical Quantization Fixes
-
TurboQuant-Optimized llama.cpp Fork Delivers GFX906 GPU Acceleration
-
Gemma 4 26B Achieves Impressive Local Performance With Proper Configuration
-
TurboQuant in Llama.cpp Achieves 6X Smaller KV Cache
-
Microsoft Quantum Development Kit Ported to Rust: 100x Faster and Smaller
-
NVIDIA and Google Optimize Gemma 4 AI Models for Local RTX Deployment
-
April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
-
Apple Silicon Macs Run Local AI Faster with Ollama's New MLX Support
-
Lotte Innovate and DeepX Collaborate on Mass Production of Domestic AI Semiconductors
-
Is Anyone Working on an AI Operating System?
-
Converting a Home Server Into a Production AI Appliance
-
Forensic Beats Mem0 with 90.1% on LOCOMO Benchmark
-
Lemonade 10.0.1 Improves Setup Process For Using AMD Ryzen AI NPUs On Linux
-
Qwen 3.5 Models: Optimal Settings and Reduced Overthinking Configuration
-
Llama.cpp ROCm 7 vs Vulkan Performance Benchmarks on AMD Mi50
-
How to Build a Self-Hosted AI Server with LM Studio: Step-by-Step Guide
-
ik_llama.cpp Fork Delivers 26x Faster Prompt Processing on Qwen 3.5 27B
-
Multi-Token Prediction support coming to MLX-LM for Qwen 3.5
-
DeepSeek R1 RTX 4090 vs Apple M3 Max: Benchmark & Performance Guide
-
Mamba 3: State Space Model Architecture Optimized for Inference
-
Custom GPU Multiplexer Achieves 0.3ms Model Switching on Legacy Hardware
-
A New Magnetic Material for the AI Era
-
Kimi Introduces Attention Residuals: 1.25x Compute Performance at <2% Overhead
-
Show HN: Merrilin.ai – Code Blocks in Your Books, Finally
-
Qwen3.5-397B Achieves 282 tok/s on 4x RTX PRO 6000 Blackwell Through Custom CUTLASS Kernel
-
Running Qwen3.5-27B Across Multiple GPUs Over LAN Achieves Practical Speed for Local Inference
-
Open-Source GreenBoost Driver Augments NVIDIA GPU VRAM With System RAM and NVMe Storage
-
Intel OpenVINO Backend Support Now Available in llama.cpp
-
Best Local LLM Models 2026: Developer Comparison
-
Llama.cpp Adds True Reasoning Budget Support
-
Cutile.jl Brings Nvidia CUDA Tile-Based Programming to Julia
-
Simple Layer Duplication Technique Achieves Top Open LLM Leaderboard Performance
-
8 Local LLM Settings Most People Never Touch That Fixed My Worst AI Problems
-
FreeBSD 14.4 Released: Implications for Local LLM Deployment
-
When Running Ollama on Your PC for Local AI, One Thing Matters More Than Most
-
Llama.cpp Prompt Processing Optimization: Ubatch Size Configuration Guide
-
Show HN: Asterode – Multi-Model AI App with Memory and Power Features
-
HyperExcel Seeks 150 Billion Won Series B to Scale LPU and Verda in Korea
-
Apple Unveils MacBook Pro With M5 Pro and M5 Max for On-Device AI
-
Intel Arc Pro B70 Workstation GPU Confirmed via vLLM AI Release Notes
-
Apple Neural Engine Reverse-Engineered for Local Model Training on Mac Mini M4
-
AMD Expands Ryzen AI 400 Series Portfolio for Consumer and Enterprise AI PC Options
-
Switch Qwen 3.5 Thinking Mode On/Off Without Model Reload Using setParamsByID
-
Qwen3.5-35B RTX 5080 Experiments Confirm KV q8_0 as Free Lunch, Q4_K_M Remains Optimal
-
LLmFit: One-Command Hardware-Aware Model Selection Across 497 Models and 133 Providers
-
Krasis: Hybrid CPU/GPU MoE Runtime Achieves 3,324 Tokens/Second Prefill on RTX 5080
-
Accuracy vs. Speed in Local LLMs: Finding Your Sweet Spot
-
Building a Privacy-Preserving RAG System in the Browser
-
DeepSeek Paper – DualPath: Breaking the Bandwidth Bottleneck in LLM Inference
-
The Complete Developer's Guide to Running LLMs Locally: From Ollama to Production
-
Qwen3.5 Thinking Mode Can Be Disabled for Production Inference Optimization
-
Show HN: A Human-Curated, CLI-Driven Context Layer for AI Agents
-
What Breaks When AI Agent Frameworks Are Forced Into <1MB RAM and Sub-ms Startup
-
Kioxia Sampling UFS 5.0 Embedded Flash Memory for Next-Generation Mobile Applications
-
Enhanced Interface Speed Enables High-Performance On-Device AI Features in Smartphones
-
Open-Source Framework Achieves Gemini 3 Deep Think Level Performance Through Local Model Scaffolding
-
Breaking the Speed Limit: Strategies for 17k Tokens/Sec Local Inference
-
Open-Source + AI: ggml Joins Hugging Face, llama.cpp Stays Open—Local AI's Long-Term Home
-
The Path to Ubiquitous AI (17k tokens/sec)
-
NVIDIA Releases Dynamo v0.9.0: Infrastructure Overhaul With FlashIndexer and Multi-Modal Support
-
GPT4All Replaces Ollama On Mac After Quick Trial
-
Cloudflare Releases Agents SDK v0.5.0 with Rust-Powered Infire Engine for Edge Inference
-
AMD Announces Day 0 Support for Qwen 3.5 LLM on Instinct GPUs
-
Scaling llama.cpp On Neoverse N2: Solving Cross-NUMA Performance Issues
-
Scaling llama.cpp On Neoverse N2: Solving Cross-NUMA Performance Issues
-
Memio Launches AI-Powered Knowledge Hub for Android with Local Processing
-
NAS System Achieves 18 tok/s with 80B LLM Using Only Integrated Graphics
-
Developer Switches from Ollama and LM Studio to llama.cpp for Better Performance
-
Carmack Proposes Using Long Fiber Lines as L2 Cache for Streaming AI Data