exo

exo is a local LLM runner for people who want to run models too large for a single machine's memory. It joins devices into a cluster and runs inference on your own hardware.

• Models across devices: Devices discover each other automatically, and exo splits model processing according to available resources and the speed of connections between machines.
• Compatible clients: Connect applications through OpenAI Chat Completions, OpenAI Responses, Claude Messages or Ollama-compatible APIs, including tools such as OpenWebUI.
• Model choice: Run models such as DeepSeek v3.1, Qwen3-235B and Kimi-K2-Thinking, or load custom models from HuggingFace.
• Cluster dashboard: Manage the cluster and chat with models through the built-in browser dashboard; a macOS app also runs in the background.

exo is open source under Apache 2.0 and runs on macOS and Linux. It uses MLX for inference, with GPU processing on macOS and CPU processing on Linux. Models download to local storage.

Tensor parallelism lets devices process parts of a model together. Compatible Macs with Thunderbolt 5 can use RDMA to reduce communication latency between devices; performance depends on the hardware and connections in the cluster. The macOS app requires macOS Tahoe 26.2 or later.