Tagged "llm-frameworks"
9 articles tagged llm-frameworks, 17 February 2026 to 2 May 2026. Newest first.
-
Google Drops COSMO: Experimental On-Device AI Assistant for Android
Google has released COSMO, a new experimental AI assistant designed for on-device processing on Android, demonstrating renewed focus on edge inference capabilities.
-
GPU Memory for LLM Inference (Part 1)
A detailed technical guide exploring GPU memory optimization strategies for running large language models efficiently during inference, critical knowledge for anyone deploying LLMs locally with limited VRAM.
-
Samsung launches Galaxy Book6 series in India with Nvidia RTX 5070 graphics and on-device AI
Samsung's new Galaxy Book6 laptops feature Nvidia RTX 5070 graphics enabling powerful on-device AI capabilities, representing mainstream hardware adoption of local AI inference.
-
Introduction to Nyreth v1.0
Nyreth v1.0 has been released with new capabilities for local LLM deployment. Video walkthrough introduces features and implementation details relevant to on-device inference practitioners.
-
Critical: LiteLLM Supply Chain Attack Detected, Bifrost Alternative Released
PyPI versions 1.82.7 and 1.82.8 of LiteLLM were compromised with credential-stealing malware. The community has compiled alternatives including Bifrost, a Go-based replacement claiming 50x faster P99 latency.
-
What AI Augmentation Means for Technical Leaders
Birgitta Boeckeler discusses practical implications of AI augmentation for engineering teams, covering deployment strategies, tool selection, and organizational considerations for AI-augmented workflows.
-
DeepSeek Paper – DualPath: Breaking the Bandwidth Bottleneck in LLM Inference
DeepSeek researchers present DualPath, a novel approach to address bandwidth limitations during LLM inference. This work tackles one of the primary performance bottlenecks in local and edge LLM deployment.
-
Enterprise Infrastructure Guide: Running Local LLMs for 70-150 Developers
A detailed discussion on designing local LLM infrastructure for agentic coding workflows across a growing development team. Covers scaling considerations, deployment architecture, and best practices for enterprise-grade on-device AI integration.
-
Self-Hosted AI: A Complete Roadmap for Beginners
KDnuggets publishes a comprehensive guide for deploying and running AI models locally, covering essential concepts, tools, and best practices for self-hosted inference. This resource serves as a practical entry point for developers new to local LLM deployment.