Tagged "optimization"
-
Tether AI Upgrades QVAC SDK With TurboQuant for Data Center-Sized Memory on Everyday Devices
-
Phison and Intel Roll Out aiDAPTIV to Boost Local AI on Intel AI PC Platforms
-
JetBrains Releases Mellum2: A 12B MoE Model for Fast, Specialized Tasks
-
Good LLM Development and Usage Patterns
-
What Apple Knows About AI That Silicon Valley Won't Admit
-
Tweaking Local Language Model Settings with Ollama
-
The Infrastructure Behind Making Local LLM Agents Actually Useful
-
MediaTek Dimensity 8550 Shifts Focus to Gemini Nano V3 and On-Device AI on Phones
-
Alibaba Cloud Joins PyTorch Foundation as Platinum Member
-
Local LLM Setup: How to Use RAG and an Embedding Model to Stop Wasting Context
-
Why AI Hardware Is a Chip Layer Problem
-
A Maintainability Ratchet for AI-Assisted Python
-
Why Your Docker Container Is 1.2GB When It Should Be 80MB
-
The Brain vs. Deep Learning Part I: Computational Complexity Analysis
-
Intel llm-scaler-vllm 1.4 Released With Updated Components and Arc Pro B70 Support
-
Google's Cormac Brick on Tiny LLMs for On-Device Agents
-
Google Tensor SDK Beta with LiteRT Enables Efficient On-Device AI
-
Google and Synaptics Partner on Coralboard for Immersive Edge AI Experiences
-
Google's Offline AI App Gets Three Major Feature Upgrades
-
Samsung's Exynos 2800 Could Be the First Mobile Chip to Use HBM for Powerful On-Device AI
-
Running Large Language Models on Single-Board Computer Clusters: Creative Edge Deployment
-
Ansede-static: Offline SAST Tool Demonstrates Value of Local AI Tools
-
Linux 7.1-rc4 Released: Kernel Updates Relevant to Local LLM Inference
-
AMD's Lemonade SDK Advances macOS Support for Local AI Inference with ROCm 7.13
-
MegaTrain: Full Precision Training of 100B+ Parameter LLMs on a Single GPU
-
HP's On-Device AI Needs More If It Is Going to Compete With Copilot
-
Google Limits Gemini Intelligence to New Flagships—Hardware Requirements for Local Deployment
-
llama.cpp Delivers Sharp Performance Gains for AMD RDNA3 Users
-
Kog AI – Building a Real-Time Inference Stack on AMD Instinct GPUs
-
Arm and Google Collaborate on On-Device AI Optimization Techniques
-
Mainline Linux 6.12 on Annapurna Labs Alpine V2 (Ubiquiti UNVR, UDM-Pro)
-
Lython: Experimental Python Compiler Toolchain Based on LLVM
-
One LM Studio Setting Change Makes Local LLMs Competitive With Cloud Models
-
DFlash Speculative Decoding Delivers 8.5x Speed Improvement for LLM Inference
-
Nota AI Partners with Mobilint to Accelerate On-Device AI on Domestic NPU Infrastructure
-
A 49-Line Physics Classifier That Beats kNN on 76% of Benchmarks
-
Google Explains Why AICore Storage Requirements Are Increasing on Android
-
Local LLMs Work Best When You're Not Loyal to Just One
-
Building a Raspberry Pi-Based Local LLM Server for Remote Access
-
New Open-Source Tool Automatically Matches Local LLMs to Your PC Hardware
-
Private LLM vs. ChatGPT: When It Makes Sense for Business
-
How Much "Brain Damage" Can an LLM Tolerate?
-
Estimating Black-Box LLM Parameter Counts via Factual Capacity
-
Wipeout Clone Runs Native on ESP32-S3, Pushing Edge Hardware to Its Limits
-
Stop Guessing: Open-Source Tool Predicts Which Local LLMs Run on Your PC
-
Blueprint: AI Hardware Design
-
Google's Gemma 4 Brings Powerful On-Device AI to Phones and Laptops
-
Using a Local LLM as a Zero-Shot Classifier
-
Building Real-World On-Device AI with LiteRT and NPU
-
Llama.cpp's Auto Fit Feature Quietly Reshapes Local AI Inference on Consumer Hardware
-
Google's Gemma 4 Finally Makes Local LLM Deployment Compelling for Practitioners
-
16 Ways to Make a Small Language Model Think Bigger
-
ZeusHammer: Built an AI Agent That Thinks Locally
-
Controlling the Secondary Fan on Minisforum AI Pro HX 370
-
llama.cpp Merges Speculative Checkpointing for Major Inference Speed Boost
-
115 TOPS in 0.67L: CHUWI AuBox X Packs On-Device AI Power Into a Palm-Sized Mini PC
-
Sorting 1M u64 KV-Pairs in 20ms on i9-13980HX Using Branchless Rust Implementation
-
Learn LLM Internals
-
A Deep Dive into Tinygrad AI Compiler
-
Users Report Significant Performance Improvements After Migrating from Ollama to llama.cpp
-
CarryAI's Serverless Vision-Language Models Enable On-Device Multimodal AI
-
PyTorch Foundation Welcomes Helion as a Foundation-Hosted Project to Standardize Open, Portable, and Accessible AI Kernel Authoring
-
Microsoft Quantum Development Kit Ported to Rust: 100x Faster and Smaller
-
NVIDIA and Google Optimize Gemma 4 AI Models for Local RTX Deployment
-
Qwen 3.6-Plus Released
-
Show HN: Extra-Platforms, Python Library to Detect OS, Arch, Shell, CI, AI
-
Local AI didn't replace my subscriptions, but it did take over these 6 tasks
-
Select the Right Hardware for Your Local LLM Deployment with This Online Guide
-
DeepSeek V3 Complete Guide: Deploy and Optimize Local AI in 2026
-
Mixed KV Cache Quantization: Performance Risks and Pitfalls
-
Linux Significantly Outperforms Windows for Local LLM Inference
-
TurboQuant Benchmarked in Llama.cpp: Google's Extreme Compression Research Tested in Practice
-
RotorQuant: 10-19x Faster Quantisation Alternative Using Clifford Algebra
-
Quantization Reveals Outliers Impacting LLM Accuracy
-
Apple Gets Full Gemini Access and Uses Distillation to Build Lightweight On-Device AI
-
RF-DETR Nano and YOLO26 Enable On-Device Object Detection on Smartphones
-
NVIDIA Releases GPT-OSS-Puzzle-88B, a Deployment-Optimized Model
-
Nota AI and SiMa.ai Partner on Physical AI Technology for Local Deployment
-
.APKs Are Just .ZIPs: Semi-Legally Hacking Software for Orphaned Hardware
-
MacinAI Local brings functional LLM inference to classic Macintosh hardware
-
AI's Impact on Mathematics Analogous to Car's Impact on Cities
-
You're Using Your Local LLM Wrong If You're Prompting It Like a Cloud LLM
-
Auto-retry Claude Code on subscription rate limits (zero deps, tmux-based)
-
India's Mobile-First AI Strategy Could Accelerate Local Inference Adoption in Emerging Markets
-
Linux 7.0 AMDGPU Fixing Idle Power Issue For RDNA4 GPUs After Compute Workloads
-
Show HN: VmExit – An Experiment in AI-Native Computing
-
Quantization Explained: Q4_K_M vs AWQ vs FP16 for Local LLMs
-
SK Hynix Completes Qualification for LPDDR6 Memory Optimized for AI Inference
-
Llama.cpp Prompt Processing Optimization: Ubatch Size Configuration Guide
-
ETH Zurich Research Challenges Context-Length Assumptions in LLM Agents
-
OpenWrt 25.12.0 – Stable Release
-
Building a Dependency-Free GPT on a Custom OS
-
Critical: Qwen 3.5 Requires BF16 KV Cache, Not FP16 for Accurate Inference
-
How to Run High-Performance LLMs Locally on the Arduino UNO Q
-
Bare-Metal LLM Inference: UEFI Application Boots Directly Into LLM Chat
-
Unsloth Dynamic 2.0 GGUFs
-
Accuracy vs. Speed in Local LLMs: Finding Your Sweet Spot
-
Snapdragon 8 Elite Gen 5 for Galaxy Official: 5 Key Improvements that Push the Boundaries
-
On-Device AI in Mobile Apps: What Should Run on the Phone vs the Cloud (A 2026 Decision Guide)
-
Extracting 100K Concepts from an 8B LLM
-
Every agent framework has the same bug – prompt decay. Here's a fix
-
DeepSeek Releases DualPath: Addressing Storage Bandwidth Bottlenecks in Agentic Inference
-
Mirai Announces $10M to Advance On-Device AI Performance for Consumer Devices
-
Advanced Quantization Techniques Show Surprising Performance Gains Over Standard Methods
-
Show HN: A Ground Up TLS 1.3 Client Written in C
-
Which Web Frameworks Are Most Token-Efficient for AI Agents?
-
Wave Field LLM Achieves O(n log n) Scaling: 825M Model Trained to 1B Parameters in 13 Hours
-
Custom Portable Workstation Optimized for Local AI Inference Builds
-
A Tool to Tell You What LLMs Can Run on Your Machine
-
Elastic Introduces Best-in-Class Embedding Models for High Performance Semantic Search
-
Breaking the Speed Limit: Strategies for 17k Tokens/Sec Local Inference
-
Yet Another Fix Coming for Older AMD GPUs on Linux – Thanks to Valve Developer
-
AI-Powered Reverse-Engineering of Rosetta 2 for Linux
-
GGML Joins Hugging Face: What This Means for Local Model Optimization
-
DietPi Released a New Version v10.1