Harbor
An open-source local LLM stack manager that connects Ollama, llama.cpp and AI apps through Docker Compose. Includes a CLI and companion app.
Harbor is a CLI and companion app for people experimenting with AI on their own hardware. It manages a local LLM development environment, connecting model backends to chat interfaces and supporting services so you don't have to configure each connection yourself. It's open source under Apache 2.0.
Docker Compose handles the container services. Backends include Ollama, llama.cpp and vLLM, with chat interfaces such as Open WebUI and LibreChat. On macOS, Docker Model Runner, MLX and oMLX can run inference directly on the host with Metal acceleration. GGUF models work through llama.cpp; the PrismML backend supports Ternary Bonsai 2 GGUFs on CPU, NVIDIA and AMD ROCm hardware.
The connections extend beyond chat. SearXNG supplies web search to Open WebUI, Perplexica and Local Deep Research. Speaches adds speech recognition and text-to-speech, while ComfyUI connects FLUX image generation to Open WebUI. MCP integrations let chat interfaces use external tools, and Dify or n8n can support larger AI workflows.
For coding, Harbor connects local OpenAI-compatible backends to installed tools including Codex, Claude Code and OpenCode. Harbor Boost adds workflows for web research, checking deliverables and reviewing scope or style before an agent answers.
Configuration profiles let you keep different model setups, and built-in benchmarks can evaluate models against your own tasks. Services are accessible over your LAN, including from a phone. Harbor can also export your selected stack as a standalone Docker Compose file.
