Tagged "benchmark-report"
70 articles tagged benchmark-report, 25 March 2026 to 18 October 2026. Newest first.
-
Modular's MAX Achieves 5.5x Performance Improvement Over vLLM on Kimi K2.5
Modular's MAX inference engine demonstrates significant performance gains, delivering 5.5x speedup compared to vLLM on Kimi K2.5, marking a major breakthrough in local LLM inference optimization.
-
Modular's MAX Framework Delivers 5.5x Speedup Over vLLM on Kimi Models
Modular's MAX inference framework achieves 5.5x performance improvement over vLLM when running Kimi K2.5 models, demonstrating significant optimisations for local LLM inference speed and efficiency.
-
Modular's MAX Inference Stack Achieves 5.5x Speedup on AMD Hardware
Modular's Dakkak reports MAX Inference delivering 5.5x faster performance on Kimi K2.5, with AMD's 30% hardware discount reshaping the economics of local model deployment.
-
vLLM on Vera Rubin Posts 7.8x GB200 Throughput per Chip
vLLM achieves 7.8x throughput improvements on NVIDIA's GB200 hardware through Vera Rubin optimizations, demonstrating significant performance gains for local inference clusters.
-
AMD ROCm vs. Vulkan Performance Benchmarking for Local AI Inference on Lemonade Server
Phoronix benchmarks AMD's ROCm and Vulkan backends for llama.cpp, comparing GPU acceleration performance for local inference on AMD hardware. The results provide practical insights for practitioners choosing between different GPU acceleration strategies.
-
Llama.cpp Achieves 2.2x Faster Inference on Intel Arc GPUs
A developer reports significant performance improvements running llama.cpp on Intel Arc graphics cards, achieving 2.2x more tokens per second through optimizations. This breakthrough demonstrates Intel's viability as a cost-effective alternative to Nvidia for local LLM inference.
-
ROCmFix and InferBench: AMD Local-LLM Setup and Vulkan vs. HIP Benchmarking
Practical tools and benchmarks for AMD GPU-based local LLM inference, comparing Vulkan and HIP backend performance to optimize inference on AMD hardware.
-
Benchmarking Local LLM Servers: Llama.cpp, Llamafile, LM Studio, and Ollama
A practical benchmark comparison of four major local LLM serving frameworks, measuring performance across speed, memory usage, and ease of deployment on consumer hardware.
-
Benchmarking Local LLM Servers: Llama.cpp, Llamafile, LM Studio, and Ollama
Mozilla AI publishes comprehensive benchmarks comparing four major local LLM inference servers, providing practical performance data for selecting the right tool for on-device deployment.
-
TensorRT Edge-LLM Achieves 6.4x Faster Performance on Jetson AGX Thor
NVIDIA's TensorRT Edge-LLM completes the MLPerf Edge Agentic Benchmark 6.4x faster on Jetson AGX Thor, demonstrating significant performance improvements for edge AI inference on specialized hardware.
-
Benchmarking Qwen3.8 27B Quantizations: 4-bit Holds Up, 1-bit Collapses
Detailed quantization benchmarks for Qwen3.8 27B reveal that 4-bit quantization maintains strong performance while extreme 1-bit quantization severely degrades output quality, providing practical guidance for practitioners choosing compression levels.
-
Benchmarking Qwen 3.8 27B Quantizations: 4-Bit Holds Up, 1-Bit Collapses
Detailed quantization benchmarks for Qwen 3.8 27B revealing how 4-bit quantization maintains model quality while 1-bit approaches fail significantly.
-
Benchmarking Qwen 3.8 27B on RTX 5090 and Beyond
Comprehensive performance benchmarking of Qwen 3.8 27B model on high-end consumer GPUs like the RTX 5090, providing practical insights for local deployment scenarios.
-
Qwen3.8 27B Quantization Benchmarks: 4-Bit Remains Optimal Trade-off
New quantization benchmarks for Qwen3.8 27B show that 4-bit quantization maintains excellent quality, while 1-bit approaches suffer significant quality collapse, providing crucial guidance for local deployment decisions.
-
Benchmarking Qwen3.8 27B Quantizations: 4-bit Shows Strong Performance, 1-bit Collapses
Detailed quantization benchmarks for Qwen3.8 27B reveal that 4-bit quantization maintains strong performance while 1-bit variants suffer significant degradation, providing practical guidance for local deployment scenarios.
-
Qwen 3.8 27B Successfully Runs on 16GB RAM Using LM Studio
Community testing confirms Qwen 3.8 27B operates efficiently on 16GB systems with LM Studio, making a capable 27-billion parameter model accessible to users with modest hardware. Quantised GGUF weights enable practical local deployment without expensive GPUs.
-
Local Model Performance Benchmarks on MacBook Pro M5 Max: Real-World Inference Metrics
Comprehensive performance testing of local LLMs on Apple Silicon M5 Max hardware reveals practical throughput and latency metrics for developers evaluating on-device inference on macOS.
-
Benchmarking Local LLMs on Consumer Hardware: Real-World Performance Data
A practical benchmark comparing local LLM performance on a typical laptop provides concrete data on inference speed, memory usage, and capabilities across different models. This real-world data helps practitioners choose appropriate models for their hardware constraints.
-
DeepSeek V4 Flash Achieves 82.7% on Terminal-Bench 2.1
DeepSeek V4 Flash demonstrates strong benchmark performance with 82.7% accuracy on Terminal-Bench 2.1 using a public harness. This efficient model variant shows promise for local deployment scenarios requiring high capability with reasonable resource constraints.
-
AMD Ryzen AI PCs Demonstrate 18 Hours Weekly Productivity Gains in Project Management Tasks
A new study shows AMD Ryzen AI-powered PCs significantly accelerate project management workflows, with users saving up to 18 hours per week on common tasks. This validates the practical benefits of on-device AI for workplace productivity without cloud dependencies.
-
Testing Top Local LLMs Against ChatGPT and Claude Reveals Performance Gaps
A comprehensive benchmark comparing leading local LLM options with commercial alternatives like ChatGPT and Claude uncovers specific use cases where open models struggle. This evaluation provides practical guidance for choosing between local and cloud-based solutions.
-
K3 Model Achieves 20 Tokens/Second on 80x RTX 5090 Cluster
Benchmark results show K3 model inference achieving 20 tokens per second across an 80-GPU RTX 5090 setup, providing insights into scaling strategies for high-throughput local deployments.
-
Nvidia Isn't the Only Choice for Local LLMs Anymore, and AMD Test Proves It
A practical benchmark demonstrates that AMD GPUs are now competitive for running local LLMs, challenging Nvidia's dominance and expanding hardware options for self-hosted inference.
-
GPT-5.6 Sol vs. Claude Fable 5 in CNC Red Alert 2 Benchmark
A new benchmark comparing frontier LLM variants in real-time strategy gameplay demonstrates practical performance evaluation methodologies. This shows how gaming environments can serve as rigorous testbeds for model reasoning and decision-making capabilities.
-
AMD Ryzen 7 7700X3D Linux Performance Review
Phoronix publishes detailed Linux performance benchmarks for the AMD Ryzen 7 7700X3D processor, providing critical data for practitioners evaluating CPU hardware for local LLM inference and edge AI workloads. The 3D V-Cache architecture offers unique advantages for memory-heavy AI tasks.
-
Cost vs. Accuracy in CursorBench 3.1: The Effect of Family and Spend
New benchmark analysis reveals cost-accuracy tradeoffs across different LLM families, providing critical insights for selecting models for local deployment based on performance requirements and resource constraints.
-
RISC-V RVV Vector Benchmarks: SpacemiT K3 SoC Performance for Edge AI
Performance benchmarking of the SpacemiT K3 system-on-chip using RISC-V vector extensions reveals competitive inference capabilities for local AI workloads on alternative CPU architectures.
-
Tiny LLM Benchmark: Jetson Orin Nano Super 8GB
Comprehensive benchmark results for running small language models on NVIDIA's Jetson Orin Nano Super with 8GB memory, providing practical performance data for edge LLM deployment.
-
DeepSWE Benchmark Updated with GLM 5.2 and Expanded Model Comparisons
The DeepSWE software engineering benchmark has been updated with new results for GLM 5.2 and other models, providing fresh performance data for evaluating local LLM deployments on code generation tasks. This comprehensive benchmark helps practitioners select appropriate models for their infrastructure.
-
Intel Core Ultra X7 Panther Lake Performance Benchmarked on Linux
Phoronix publishes comprehensive performance benchmarks for Intel's newest Core Ultra X7 Panther Lake processors running on Linux 7.1. These results are critical for evaluating local LLM inference performance on current-generation Intel hardware.
-
RTX 5080 and RTX 3090 Setup Achieves 80 Tok/s on Qwen 3.6 27B Q8
A practical benchmark demonstrating impressive inference throughput using dual NVIDIA GPUs running quantized Qwen 3.6 27B model. This setup showcases real-world performance metrics for local LLM deployment on consumer-grade hardware.
-
vLLM vs Ollama 2026: 793 vs 41 TPS Performance Benchmark
A comprehensive benchmark comparison reveals vLLM achieves 793 tokens per second versus Ollama's 41 TPS, highlighting a significant 19x performance gap for local LLM inference workloads.
-
LLM Memory Systems Benchmark: High Recall, Near-Zero Precision for Tested Systems
A new benchmark reveals critical weaknesses in LLM memory systems, showing high recall but near-zero precision across tested implementations. This finding is crucial for developers building stateful local LLM applications and agentic systems.
-
vLLM vs Ollama 2026: Performance Benchmark Reveals 9x Throughput Gap
A comprehensive benchmark comparison shows vLLM significantly outperforming Ollama in throughput metrics, with implications for choosing the right inference framework for local deployments.
-
M5 Max MacBook Runs Local Large Language Models Efficiently
Testing demonstrates that Apple's M5 Max processor effectively handles local large language model inference with strong performance characteristics. The MacBook's unified memory architecture proves particularly well-suited for efficient LLM execution without dedicated accelerators.
-
A/B Tested Gemini 3.1 Pro vs. Claude Opus 4.6 – Usage Quota and Quality Comparison
A detailed comparative benchmark between Gemini 3.1 Pro and Claude Opus 4.6 examines usage quotas and output quality, providing practical insights for practitioners evaluating cloud versus local inference trade-offs. The analysis highlights cost-effectiveness and performance considerations when choosing between commercial APIs and self-hosted solutions.
-
110 Tokens/Second on RTX 4070 Super with Qwen 3.6 35B
A significant performance benchmark demonstrates that consumer-grade GPUs can achieve excellent inference speeds with optimized models, enabling practical local deployment of 35B parameter models.
-
Hardware LLM Taalas Reaches >14,000 TPS on Llama 3.1 8B
Taalas demonstrates breakthrough throughput of over 14,000 tokens per second on Llama 3.1 8B, showcasing specialized hardware acceleration for local and edge LLM deployment.
-
Benchmarking a Portable AI Workstation: Lenovo ThinkPad P16 Gen 3, Part 2
Detailed performance analysis of the Lenovo ThinkPad P16 Gen 3 as a portable AI workstation, providing real-world benchmarks for local LLM inference and training workflows.
-
Bito's AI Architect Improves Claude Opus Task Success Rate by 35%
Bito has demonstrated a 35% improvement in Claude Opus's task success rate on SWE-Bench Pro through their AI Architect framework. This benchmark shows significant gains in model capability for code-related tasks.
-
ROCm 7.2.3 Delivers Performance Improvements Over 7.0.0 on AMD Radeon AI PRO
Phoronix benchmarks show measurable performance gains with ROCm 7.2.3 compared to version 7.0.0 on AMD's Radeon AI PRO R9700 GPU. The improvements highlight the importance of staying current with driver and runtime updates for optimal local inference performance.
-
$200 NVIDIA V100 Server GPU Mod Beats RTX 3060 in Local LLM Test
A creative hardware modification using refurbished NVIDIA V100 server GPUs demonstrates strong price-to-performance for local LLM inference, outperforming newer consumer-grade GPUs at a fraction of the cost.
-
Small On-Device AI Model Beats Claude Sonnet 4.5 and GPT-5
A newly optimized on-device AI model demonstrates performance that exceeds leading cloud-based models on specific benchmarks. This breakthrough challenges assumptions about model size and cloud superiority for local deployment.
-
NIST's CAISI Evaluation of DeepSeek V4 Pro Finds It On Par with GPT-5
NIST's comprehensive evaluation framework reveals that DeepSeek V4 Pro achieves performance parity with GPT-5 on standardized benchmarks, with implications for local deployment viability.
-
Xmemory: Benchmarking Structured AI Memory Against RAG and Hybrid RAG
A new benchmark comparing structured AI memory systems against retrieval-augmented generation (RAG) approaches, providing insights for optimizing local LLM deployments with better context management and memory efficiency.
-
Linux Crushes Windows on llama.cpp Inference by Double Digits
New benchmarks reveal significant performance advantages for llama.cpp inference on Linux systems compared to Windows, with improvements reaching double-digit percentages across various model sizes.
-
LLMs Consume 5.4x Less Mobile Energy Than Ad-Supported Web Search
Research demonstrates that local LLM inference uses significantly less energy than cloud-based web search on mobile devices, highlighting a major efficiency advantage for on-device deployment.
-
Speculative Decoding Achieves 29% Speed Boost for Gemma-4 31B
Benchmarks show speculative decoding with Gemma-4 E2B draft model delivers 29% average throughput improvement and 50% gains on code tasks. This practical optimization technique significantly accelerates local inference on consumer GPUs.
-
MiniMax-M2.7 Delivers Exceptional Performance on Consumer Hardware
MiniMax-M2.7 benchmarks show strong throughput (127.7 tok/s on dual RTX PRO 6000 Blackwell) and efficient VRAM utilization, positioning it as a practical alternative to larger models for resource-constrained deployments.
-
Gemma 4 31B vs Qwen 3.5 27B: Comprehensive Long Context Benchmark
Community benchmark comparing Gemma 4 31B and Qwen 3.5 27B for long context workloads on 24GB VRAM, establishing these as the top local models for mid-range GPU setups.
-
Intel Arc Pro B70 32GB Achieves 12 Tokens/Sec on Qwen 3.5-27B
Intel Arc Pro GPU hardware demonstrates strong performance running Qwen 3.5 27B quantized models with vLLM and llama.cpp, establishing alternative hardware viability for local deployment.
-
Qwen 3.5 122B Achieves 198 Tokens/sec on Dual RTX PRO 6000 Blackwell GPUs
A detailed optimization case study demonstrates running Qwen 3.5 122B at impressive inference speeds on a budget dual-GPU Blackwell setup. The community shares verified benchmarks with full methodology and reproducible results for large-scale local deployment.
-
Warp Decode vs. vLLM's Triton Kernel: Performance Crossover Analysis
A detailed technical comparison analyzing where Warp Decode and vLLM's Triton kernel each excel for local LLM inference, with implications for choosing the right decoding strategy for your hardware.
-
Mano-P: Open-Source On-Device GUI Agent, #1 on OSWorld Benchmark
Mano-P, an open-source GUI agent optimized for local deployment, achieved top performance on the OSWorld benchmark, demonstrating state-of-the-art capabilities for on-device automation tasks.
-
Gemma 4 Achieves Top Multilingual Performance Across European Languages
Benchmarks show Gemma 4 31B ranking among the best models for European languages including Danish, Dutch, French, Italian, and Finnish, offering strong multilingual support for local deployment scenarios.
-
Show HN: Willitrun – Check if Any ML Model Runs on Any Device (Benchmark-Backed)
Willitrun is a new tool that helps developers determine whether specific machine learning models can run on particular devices, backed by real benchmarking data to guide local deployment decisions.
-
Comprehensive Benchmark: 37 LLMs Tested on MacBook Air M5 With Open-Source Tool
A detailed benchmark study evaluating 37 language models across 10 families on Apple's M5 MacBook Air, complete with open-source benchmarking tool for community replication and testing on Mac hardware.
-
Quantization Strategy Comparison: Balancing Quality and Speed on Consumer Laptops
Detailed benchmarking of different GGUF quantization methods for Qwen 3.5 4B on Intel Lunar Lake iGPU reveals optimal compression strategies for small model deployment on resource-constrained hardware.
-
Gemma 4 31B Achieves Third Place on FoodTruck Bench, Beating Larger Models
Google's Gemma 4 31B model has demonstrated exceptional performance on the FoodTruck Bench, ranking third and outperforming significantly larger models like GLM 5 and Qwen 3.5 397B. The result highlights major improvements in long-horizon task handling for locally deployable models.
-
YC-Bench: GLM-5 Matches Claude Opus 4.6 at 11× Lower Cost
A new benchmark puts 12 LLMs through a year-long simulated startup experience, revealing that GLM-5 delivers comparable performance to Claude Opus 4.6 at significantly lower inference cost, enabling more efficient local deployment.
-
Gemma 4 31B Outperforms GLM 5.1 in Real-World Testing
Community benchmarks show Gemma 4 31B delivering superior performance compared to GLM 5.1, with particularly strong results in reasoning and creative text analysis tasks on consumer hardware.
-
April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
A community-contributed quick-start guide documents practical steps for deploying Gemma 4 on Mac mini hardware using Ollama, providing a reference implementation for local inference setup.
-
Qwen 3.5-27B Demonstrates Superior Performance vs Gemini 3.1 Pro and GPT-5.3
Community benchmarks show Qwen3.5-27B outperforming larger closed-source models in practical scenarios, particularly for code tasks. The open model's availability and performance characteristics make it an attractive option for local deployment when considering capability-per-resource tradeoffs.
-
Forensic Beats Mem0 with 90.1% on LOCOMO Benchmark
Forensic memory system achieves 90.1% on the LOCOMO benchmark, outperforming Mem0 and demonstrating new capabilities for local context and memory management in LLM applications.
-
M5 Max Delivers 1.7x Faster Inference Than M3 Max on Qwen 3.5 Models
Comprehensive benchmarks comparing Apple's M5 Max and M3 Max chips show significant performance gains across Qwen 3.5 model variants (27B dense, 35B MoE, 122B MoE), with the newer chip delivering 1.4x to 1.7x faster token generation using the oMLX framework.
-
Comparison of Two Frameworks: 40% Token Efficiency Improvement
A detailed comparison shows that Wasp achieves the same application functionality with 2.5M tokens versus 4.0M tokens in Next.js, highlighting the importance of framework choice for optimizing local LLM inference costs.
-
TurboQuant Benchmarked in Llama.cpp: Google's Extreme Compression Research Tested in Practice
Community members benchmarked Google's TurboQuant extreme compression technique within llama.cpp, providing practical performance data on the quantisation method. Results show how the research translates to real-world inference speed and memory usage improvements.
-
Qwen 3.5 27B Achieves 1.1M Tokens/Second on B200 GPUs with Optimized vLLM Config
A developer optimized Qwen 3.5 27B to reach 1.1 million tokens per second on 96 B200 GPUs using vLLM, with detailed configurations and all settings published on GitHub. Key optimizations included distributed parallelism, reduced context windows, FP8 KV cache, and speculative decoding.
-
Real-World Benchmark: DeepSeek-V3 Matches Claude Sonnet on Routine Coding Tasks
A practical benchmark comparing DeepSeek-V3 against Claude Sonnet on 50 real coding tasks shows DeepSeek-V3 achieving comparable quality while enabling local deployment and inference cost savings.
-
Llama.cpp Benchmark: RTX 5090 vs Enterprise Systems Compared
Comprehensive llama-bench benchmarks comparing RTX 5090 consumer GPU against DGX Spark and AMD AI395 in real-world local inference scenarios, with ROCm and Vulkan results included.