LocalAI

A self-hosted AI runtime with an OpenAI-compatible API. It runs models on CPUs or GPUs and keeps inference on your own hardware.

LocalAI runs language models, speech, vision and image generation on hardware you control. It's for developers and teams that want a self-hosted AI server for their apps without sending model requests to a cloud service. Its OpenAI-compatible API works with existing clients, and it also accepts Anthropic, Ollama and ElevenLabs API calls.

The same runtime handles chat, transcription, speech synthesis, object detection, video and 3D workloads. A GPU is optional. LocalAI supports CPU-only machines as well as NVIDIA, AMD, Intel and Apple Silicon acceleration. It runs on macOS and Linux, in Docker or Podman containers, and on Kubernetes.

LocalAI can use different engines for different models, including llama.cpp, vLLM, SGLang and MLX. Its small core obtains backends as models need them. That approach gives teams a way to serve different kinds of AI work through one API without bundling every engine into the base runtime. It supports GGUF models, including the project's APEX quantizations, which are designed to fit larger models into less GPU memory.

The integrated web interface and model gallery provide another way to work with models. For heavier use, distributed mode can place requests across machines according to available hardware and where a model is loaded. LocalAI is open source under the MIT License. Its built-in agent can use tools, MCP servers and skills, and asks for approval before running commands that change state.

Kubernetes and Cloud-Native AI LLM Gateways and Routers Inference Servers and Clusters Local Model Runners macOS Linux Docker Web Code execution Human approval llama.cpp backend MCP MLX Multi-agent workflows OpenAI-compatible API RAG

Are you the owner or maker of LocalAI?

Claim this page with an email at localai.io. LocalAI gets the verified badge, and you can upgrade the listing to be featured on localhosted. Proud to be listed? Put our badge on your site.

Want more people to find LocalAI?Promote it

Something wrong or outdated on this page?

Similar to LocalAI

View all tools

exo

1 video

An open-source local LLM runner for macOS and Linux that splits models across devices and works offline with downloaded models. Apache 2.0 licensed.

47.7KUpdated 1 month agoApache-2.0

macOS · Linux · Web#Distributed execution#Hugging Face integration#MLX

exo is a local LLM runner that combines your devices into a cluster, letting you use models too large for one machine's memory. It's for people who want to run large models on their own hardware and developers connecting existing AI clients to local inference. It runs on macOS and Linux under the Apache 2.0 license.

GPUs, Clusters and EdgeInference Servers and ClustersLocal Model Runners

LLMKube

Self-hosted LLM inference for Kubernetes with NVIDIA, AMD and Apple Silicon support, OpenAI-compatible APIs, and an Apache 2.0 license.

223Updated 16 hours agoApache-2.0

macOS · Linux#GGUF#Git integration#Guardrails

LLMKube is a free, open-source Kubernetes operator for teams and homelab owners running local LLM inference across their own hardware. It manages Linux GPU servers and Apple Silicon Macs together, so a mixed fleet can serve models through the same platform. It uses the Apache 2.0 license.

Autonomous Coding AgentsGPU Management and MonitoringInference Servers and ClustersKubernetes and Cloud-Native AILLM Gateways and Routers

OpenLLM

An open-source LLM server that runs locally or in the cloud, with OpenAI-compatible APIs, a chat UI, and an Apache 2.0 license.

12.5KUpdated 4 months agoApache-2.0

Docker · Web#Hugging Face integration#OpenAI-compatible API

OpenLLM is a self-hosted LLM server for developers who want to connect their applications to models running on their own hardware or servers. Its OpenAI-compatible API works with clients built for that interface, including the OpenAI Python client and LlamaIndex. The project is open source under the Apache License 2.0.

Inference Servers and ClustersKubernetes and Cloud-Native AILocal Model RunnersSelf-Hosted Chat Web UIsTerminal Chat Clients

Distributed Llama

An open source local LLM runner that splits inference and memory across your computers. Runs on Linux, macOS and Windows under the MIT license.

3.1KUpdated 3 months agoMIT

macOS · Windows · Linux#Distributed execution#Hugging Face integration#Quantization

Distributed Llama runs a local LLM across several computers, sharing both the computation and the model's memory use. It's for people who want to use their own networked hardware for inference rather than keep the entire workload on one machine. The C++ project is open source under the MIT license.

GPUs, Clusters and EdgeInference Servers and ClustersLocal Model RunnersTerminal Chat Clients

KubeAI

Self-hosted AI inference operator for Kubernetes with vLLM, Ollama and an OpenAI-compatible API. Runs on CPUs, GPUs or TPUs under Apache 2.0.

1.3KUpdated 1 day agoApache-2.0

Web#LoRA#Multimodal input#Ollama integration

KubeAI is an open source Kubernetes operator for teams serving AI models on their own infrastructure or cloud clusters. It manages model servers and scales them with demand, including starting from zero running replicas. It uses the Apache 2.0 license and can run on CPUs, GPUs or TPUs, including in a local Kubernetes cluster.

Embedding ServersInference Servers and ClustersKubernetes and Cloud-Native AILLM Gateways and RoutersSpeech-to-Text and Transcription

vLLM Production Stack

A self-hosted LLM serving stack built on vLLM, with an OpenAI-compatible API, request routing and GPU cluster monitoring. Licensed under Apache 2.0.

2.6KUpdated 20 hours agoApache-2.0

Web#OpenAI-compatible API#Prompt caching

vLLM Production Stack is an open source inference stack for teams serving LLMs on their own Kubernetes GPU clusters. It brings request routing and monitoring around vLLM, so applications can move from one serving instance to a distributed deployment without changing their code. It requires a GPU-enabled Kubernetes environment.

Inference Servers and ClustersKubernetes and Cloud-Native AILLM Gateways and RoutersLLM Observability and Tracing