Ask HN: What are you using for LLM inference in production?

This active discussion thread captures the current landscape of local LLM inference in production environments. Practitioners are sharing their choices of inference frameworks, optimization techniques, and hardware configurations they're using to deploy models at scale.

For local LLM practitioners, this is valuable reconnaissance on what's actually working in production versus what's theoretical. The thread typically reveals which frameworks (llama.cpp, vLLM, Ollama, MLX) are seeing real adoption, what quantization strategies are being deployed, and which hardware configurations (GPUs, CPUs, TPUs) practitioners are finding cost-effective. Understanding production patterns helps new deployments avoid common pitfalls.

These community-driven insights are particularly useful because they represent unfiltered experience from people running actual workloads, not marketing materials. Check the full discussion for practical deployment wisdom.

Source: Hacker News · Relevance: 9/10