Hosting Open-Weight Models on a Local CPU Kubernetes Cluster with vLLM
Every guide to self-hosting LLMs starts the same way: “provision a GPU node pool.” But most of us have a homelab, a company VM, or a local cluster full of ordinary CPUs. The good news: for the right models, CPU inference is genuinely useful — chatbots, summarization, classification, RAG over modest corpora. The bad news: you have to be realistic about model size, and you have to tune for CPU instead of copy-pasting GPU configs. ...