LLMKube
Kubernetes operator for self-hosted LLM inference with pluggable runtimes (llama.cpp, vLLM, TGI, Ollama, vllm-swift), multi-GPU sharding, NVIDIA CUDA + Apple Silicon Metal support, and OpenAI-compatible API.
- GitHub stars
- 232
- License
- Apache-2.0
- Latest release
- v0.10.2
7 October 2026 - Last commit
- 13 October 2026
- Runs on
- Go, Docker, Kubernetes
Commits per month, last 12 months
1,067 commits
How to self-host LLMKube
LLMKube runs on Go, Docker and Kubernetes. The project's own install guide is the place to start; most people put it on a small server with Docker and a reverse proxy for HTTPS.
- Get a server: a home server, a VPS, or a managed app host.
- Install it the way the project's docs describe, usually with Docker Compose.
- Put it behind HTTPS and set up backups before you add real data.
- Update it regularly. Watch the project's releases.
Similar apps
- Ollama
Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models.
- Open-WebUI
User-friendly AI Interface, supports Ollama, OpenAI API.
- LobeHub
Modern design AI chat framework supporting multiple AI providers, one click install MCP Marketplace and Artifacts / Thinking.
- AnythingLLM
All-in-one desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more.
- LocalAI
Run your AI models locally and generate images and audio (alternative to OpenAI and Claude).
- LibreChat
Enhanced ChatGPT-compatible AI chat interface supporting multiple AI providers, with multi-user auth, message search, and plugin support.
Questions
Is LLMKube free?
Yes. LLMKube is open source under the Apache-2.0 license, so you can use it without paying for a license. You pay only for the server it runs on.
Can I self-host LLMKube?
Yes. LLMKube runs on Go, Docker and Kubernetes. Follow the install guide in the project's documentation, or use a host that installs it for you.
Is LLMKube still maintained?
The last change to LLMKube's code was on 13 October 2026, with 1067 commits in the last 12 months.
Topics
App data from awesome-selfhosted (CC BY-SA 3.0), stars and activity from GitHub.