Tagged "arm"
38 articles tagged arm, 11 February 2026 to 29 September 2026. Newest first.
-
Forlinx Launches 20-TOPS M.2 AI Accelerator With PCIe Cascading Support
Forlinx announces a new M.2-form-factor AI accelerator offering 20 TOPS of inference performance with support for PCIe cascading, enabling scalable local LLM inference on edge devices. The compact form factor and cascading capability make it suitable for heterogeneous edge computing deployments.
-
Oh My Pi Adds Custom Model Support via vLLM, Llama.cpp, and SGLang
A new guide demonstrates running custom quantized models on Raspberry Pi using multiple inference engines including vLLM, Llama.cpp, and SGLang. This enables practical multi-engine inference workflows on edge devices with detailed configuration examples.
-
Optimizing On-Device Inference for Apple Silicon
Perplexity publishes a comprehensive guide on optimizing LLM inference specifically for Apple Silicon, covering techniques to maximize performance and efficiency on Apple's ARM-based processors for local deployment.
-
llama.cpp Build b10581 Adds DSpark Support for Faster Local Inference
The latest llama.cpp release includes native support for DSpark model optimization, enabling users to run DSpark-optimized models like LFM2.5 with maximum efficiency. This update extends llama.cpp's lead as the fastest local inference engine.
-
Llama.cpp Release b10485: GGML Sync with Platform-Specific Optimizations
Latest llama.cpp build includes GGML syncs and platform-specific improvements across macOS Apple Silicon, Intel x64, Linux ROCm, and iOS, maintaining the project's rapid release cadence for inference optimization.
-
Meta's Muse Glimmer Now Available Across All Platforms via Ollama
Ollama v0.32.8 brings Meta's Muse Glimmer to all platforms with optimized support, including state-of-the-art Apple Silicon performance via MLX. Muse Glimmer powers coding agent applications and personal assistants entirely on-device.
-
No Wi-Fi, No Data Transfer, Tablets Can Now Summarise Sensitive Documents Locally
Tablets can now process and summarize sensitive documents entirely on-device without requiring internet connectivity or data transfer. This advancement demonstrates practical deployment of LLMs on mobile hardware for enterprise document processing.
-
Arm China Unveils "Tianxuan" CPU and Xingchen 300 Platform, Targeting Ubiquitous AIoT with On-Device AI Portfolio
Arm China announced the Tianxuan CPU and Xingchen 300 platform specifically architected for on-device AI inference across IoT and edge devices in the Asian market.
-
On-Device AI vs Cloud AI: Which One Should Power Your Next Phone?
A comprehensive analysis comparing on-device versus cloud-based AI for smartphone applications, examining latency, privacy, cost, and practical trade-offs. The verdict increasingly favors hybrid approaches with local processing for common tasks.
-
Google Demonstrates New On-Device AI Features for Pixel 10
Google has unveiled new on-device AI capabilities for the upcoming Pixel 10, showcasing advances in edge inference that run directly on mobile hardware without cloud connectivity. These features highlight the industry's momentum toward practical local LLM deployment on consumer devices.
-
RISC-V RVV Vector Benchmarks: SpacemiT K3 SoC Performance for Edge AI
Performance benchmarking of the SpacemiT K3 system-on-chip using RISC-V vector extensions reveals competitive inference capabilities for local AI workloads on alternative CPU architectures.
-
Samsung Unveils UFS 5.0 Storage Solution Optimized for On-Device AI
Samsung's new UFS 5.0 storage technology delivers 10 GB/s speeds designed to eliminate I/O bottlenecks in on-device AI inference. The faster storage directly supports local model execution on flagship smartphones and edge devices.
-
Contrail Compute AIX: First RISC-V AI Execution Platform
Epic Semiconductors introduces Contrail Compute AIX, the first AI execution platform built on RISC-V architecture, expanding hardware options for local and edge AI inference beyond traditional x86 and ARM.
-
NVIDIA Joins Windows on Arm Ecosystem, Driving Arm-Based AI Notebook Adoption to 34.2% by 2029
NVIDIA has officially joined the Windows on Arm ecosystem, signaling a major shift toward Arm-based processors for local AI inference on notebooks. Industry projections suggest Arm-based AI notebooks will capture over one-third of the market by 2029.
-
Qualcomm Reveals Snapdragon C with Advanced On-Device AI Engine
Qualcomm announces Snapdragon C processor featuring a 6nm process, optimised core configuration, and dedicated on-device AI accelerator. The chip targets mobile and edge devices for local AI inference.
-
Microsoft and Nvidia to Unveil First Windows PCs with Nvidia CPUs and AI Capabilities
Microsoft and Nvidia are collaborating to introduce Windows PCs powered by Nvidia CPUs with integrated AI capabilities for local inference. This partnership signals major hardware vendors' commitment to on-device AI performance.
-
Why AI Hardware Is a Chip Layer Problem
On-device AI deployment requires fundamental hardware redesigns at the chip level, with implications for how local LLM inference will be optimized across consumer devices.
-
Local LLM Takes Control of Video Doorbell—The Future of Smart Cameras
A developer successfully deployed a local LLM to power video doorbell intelligence without cloud connectivity, demonstrating practical edge inference for smart home devices. This showcases how on-device AI can enable real-time processing while maintaining privacy.
-
Arm and Google Collaborate on On-Device AI Optimization Techniques
Arm and Google have published guidance on accelerating on-device AI inference, focusing on optimization strategies for edge devices and resource-constrained environments. The collaboration provides practical approaches for deploying LLMs efficiently on mobile and embedded systems.
-
Running AI Models Locally on M4 Processors with 24GB Memory
A technical guide explores deploying language models on Apple M4 devices with 24GB unified memory, demonstrating Apple Silicon's capabilities for local inference. The approach leverages frameworks optimized for ARM architecture and unified memory access.
-
Mainline Linux 6.12 on Annapurna Labs Alpine V2 (Ubiquiti UNVR, UDM-Pro)
New Linux kernel support for Annapurna Labs Alpine V2 processors enables more advanced edge devices to run local LLM inference with improved hardware compatibility.
-
Building a Raspberry Pi-Based Local LLM Server for Remote Access
A developer successfully deployed a local LLM server on a Raspberry Pi with remote access capabilities, demonstrating viable edge inference on minimal hardware.
-
Building a Remote-Accessible Local LLM Server on Raspberry Pi
A practical guide demonstrating how to deploy and access a local LLM server running on a Raspberry Pi from anywhere, combining edge deployment with convenient remote access.
-
Google's Gemma 4 Brings Powerful On-Device AI to Phones and Laptops
Google announces Gemma 4, an optimized model family designed specifically for efficient on-device inference on consumer hardware. This release demonstrates the industry-wide shift toward practical edge AI deployment.
-
Running a 1.7B Parameters LLM on an Apple Watch
A developer successfully deployed a 1.7 billion parameter language model on an Apple Watch, demonstrating extreme edge inference capabilities on ultra-constrained wearable hardware.
-
PyTorch Foundation Welcomes Helion as a Foundation-Hosted Project to Standardize Open, Portable, and Accessible AI Kernel Authoring
The PyTorch Foundation has incorporated Helion as a hosted project, advancing standardized kernel development for open, portable AI inference. This initiative improves the foundation for optimizing local model deployment across diverse hardware.
-
Gemma 4 on Arm: Optimized On-Device AI for Mobile and Edge Deployment
Arm releases optimizations for Gemma 4 enabling efficient deployment on Arm-based processors for mobile devices and edge endpoints, bringing enterprise-grade AI to mobile platforms.
-
Google Launches Gemma 4 Open Models for Local On-Device AI
Google releases Gemma 4, a family of open-source models built on Gemini 3 technology, optimized for local and on-device deployment across smartphones, PCs, and edge devices under an Apache 2.0 license.
-
Show HN: Extra-Platforms, Python Library to Detect OS, Arch, Shell, CI, AI
Extra-Platforms is a Python utility library that detects operating systems, architectures, CI environments, and AI frameworks—providing crucial metadata for cross-platform local LLM deployment scripts and tools.
-
Apple Plans Slimmed-Down Gemini Models for Local iPhone AI Features
Apple is reportedly adapting Google's Gemini models for on-device execution on iPhones, demonstrating enterprise-scale commitment to local LLM deployment on mobile devices.
-
Four Raspberry Pi AI Tools You Can Try This Week Beyond OpenClaw
A curated collection of practical AI tools optimized for Raspberry Pi deployment, expanding options for developers working with resource-constrained edge devices. This roundup helps practitioners identify the best tools for their specific local inference use cases.
-
Apple M4 iPad Air Targets AI Users with Double M1 Speed Performance
Apple introduces the M4 chip in iPad Air at $599, doubling M1 performance and enabling sophisticated on-device AI inference. The affordable entry point democratizes local LLM deployment on Apple hardware.
-
Qualcomm Launches Snapdragon Wear Elite for On-Device AI on Wearables
Qualcomm unveiled the Snapdragon Wear Elite chip at MWC 2026, bringing dedicated on-device AI capabilities to smartwatches and wearables. This represents a significant upgrade in edge inference capabilities for constrained devices.
-
Qwen3.5-35B Successfully Runs on Raspberry Pi 5 at 3+ Tokens/Second
Demonstration of Qwen3.5-35B inference on Raspberry Pi 5 (16GB and 8GB variants) achieving over 3 tokens/second, proving high-capacity models viable on edge devices.
-
AI-Powered Reverse-Engineering of Rosetta 2 for Linux
New project uses AI to reverse-engineer Apple's Rosetta 2 translation layer for Linux systems, potentially enabling ARM-optimized LLM inference on Linux platforms.
-
AI Is Stress Testing Processor Architectures and RISC-V Fits the Moment
RISC-V architecture emerges as a compelling alternative for AI workloads as traditional processor designs face thermal and efficiency challenges under LLM inference loads, opening new possibilities for local deployment on custom silicon.
-
Scaling llama.cpp On Neoverse N2: Solving Cross-NUMA Performance Issues
Deep dive into optimizing llama.cpp performance on ARM Neoverse N2 processors, addressing critical NUMA topology challenges for better local inference scaling.
-
Arm SME2 Technology Expands CPU Capabilities for On-Device AI
Samsung and Arm announce SME2 technology that significantly enhances CPU performance for local AI inference, potentially reducing reliance on dedicated AI accelerators.