Tagged "towards-data-science"
11 articles tagged towards-data-science, 16 April 2026 to 29 July 2026. Newest first.
-
How Much Does a Local LLM Actually Cost to Run? Energy Costs Measured on Apple Silicon
A detailed analysis quantifies the actual power consumption and operational costs of running local LLMs on Apple Silicon hardware, providing practical benchmarks for cost-conscious deployment decisions.
-
How To Build Your Own LLM Runtime From Scratch
A comprehensive guide on constructing custom LLM inference runtimes, providing practitioners with deep knowledge to optimize and control local model deployment without relying on black-box frameworks.
-
How Much Does It Actually Cost to Run a Local LLM? (Euros per Million Tokens, Measured)
A practical analysis measuring the actual operational costs of running local LLMs in euros per million tokens, providing real-world benchmarks for self-hosted inference economics.
-
How to Choose Between Small and Frontier Models
A comprehensive guide comparing trade-offs between small quantized local models and large frontier models, helping practitioners make informed deployment decisions based on latency, cost, and accuracy requirements.
-
Building Tool-Using Agents With Local LLMs
A guide on transforming local language models into autonomous agents capable of tool use and function calling. This bridges the gap between basic inference and practical agentic applications running entirely on-device.
-
Build Your Own Local AI Coding Agent with Gemma 4 and OpenCode
A practical guide to building a local AI coding agent using Google's Gemma 4 model and OpenCode framework, enabling developers to run code generation tasks entirely on-device without cloud dependencies.
-
Prefill Once, Fan Out: KV Snapshot Sharing for Multi-Agent LLM Pipelines
Towards Data Science published research on KV snapshot sharing optimization that enables efficient multi-agent LLM pipelines by reusing computed key-value caches across multiple agents. This technique significantly reduces compute requirements for local deployment scenarios.
-
The Infrastructure Behind Making Local LLM Agents Actually Useful
A comprehensive guide examining the architectural and infrastructure requirements for deploying functional local LLM agents, covering practical considerations beyond raw model performance.
-
Using a Local LLM as a Zero-Shot Classifier
Detailed guide demonstrating how to leverage locally-running language models for zero-shot text classification tasks without fine-tuning, reducing infrastructure costs and inference latency.
-
Developer Replaced GPT-4 with a Local SLM and CI/CD Pipeline Stability Improved
A Towards Data Science article documents a successful case study where replacing cloud-based GPT-4 calls with local small language models improved CI/CD pipeline reliability and reduced operational costs. This practical demonstration proves the value of local deployment for production systems.
-
Prefill Is Compute-Bound, Decode Is Memory-Bound: Optimizing GPU Utilization for LLM Inference
A deep dive into why GPUs shouldn't handle both prefill and decode phases equally, and how understanding this fundamental bottleneck can dramatically improve local LLM inference performance.