LocalAI

LocalAI is a self-hosted AI runtime for developers and teams that want to run models on their own hardware. It brings language, speech, vision and media generation into one server that existing AI applications can connect to.

• API compatibility: Use clients built for OpenAI, Anthropic, Ollama or ElevenLabs APIs with models running on your infrastructure.
• Voice and media: Run live speech conversations, transcription, speech synthesis and voice cloning, alongside image, video, music and sound generation.
• Vision tools: Detect objects, recognize faces, estimate depth and reconstruct 3D scenes from photos.
• Built-in agents: Let agents use tools, retrieve information from your data and connect to MCP servers, plugins and skills, with approval controls for shell commands.
• Multiple machines: Distribute model requests across a cluster with hardware-aware routing and failover.

LocalAI is open source under the MIT license. It runs on macOS and Linux, in Docker or Podman containers, and on Kubernetes. A GPU is optional; supported acceleration includes NVIDIA, AMD, Intel, Apple Silicon and Vulkan.

Model backends include llama.cpp, vLLM, SGLang, whisper.cpp and MLX. It supports GGUF models, including APEX quantizations intended to reduce memory requirements. An integrated web interface and model gallery provide access to models, while inference runs on hardware you control.