Apps to Run LLMs Locally

llama.cpp is a local LLM inference engine for developers building AI applications or running models on hardware they control. It supports language and vision-language models, with command-line tools and a server API for using the engine directly or connecting other software.

Qube is a desktop AI assistant for people who want to chat with models and work with private documents on their own hardware. It combines a document library with persistent memory and research tools that provide citations you can inspect.

Harbor helps people experiment with local LLMs by connecting model backends, chat interfaces and supporting services. Its CLI and companion app manage those connections in a shared development environment.

Pico MLX Server is a macOS app for people who want to run MLX models locally and use them with an existing AI chat client. It gives MLX Server a graphical interface and exposes local models through an OpenAI-compatible API. The project is open source under the MIT license and requires macOS Sonoma or later.

PowerInfer is a local LLM inference engine for developers and researchers running large models on a PC with a consumer GPU. Its performance gains depend on sparse models, which use only part of their network for each input.

LocalAI is a self-hosted AI runtime for developers and teams that want to run models on their own hardware. It brings language, speech, vision and media generation into one server that existing AI applications can connect to.

Distributed Llama runs language models across your own networked computers. It's for people who want to share inference work and model memory across several devices instead of fitting the whole workload on one machine.

Olares is a self-hosted personal cloud OS for individuals and small teams that want AI agents, models and apps on hardware they own. It brings local AI and everyday file storage into a shared workspace you can access through a browser.

Skales is an AI agent for people who want help with files, browsing, email and coding on their own devices. It can keep working on a goal after you close the chat and resume unfinished goals after a restart.

llamafile packages a language model and its runtime in a single executable for people who want to run models locally or share a ready-to-run model with others. Recipients don't need to install a separate runtime.

Unsloth is a desktop app for people who want to chat with models, train them on their own data, and create media on their own hardware. Its no-code interface brings these tasks into one workspace.

Atomic Chat is a local LLM app for people who want to chat with models on their own device or use them with coding assistants and AI agents. It began as a fork of Jan by Menlo Research.

MLX VLM is a Python package for developers and researchers who want to run multimodal models on their own Mac. It supports local inference and fine-tuning through MLX.

Jan is a desktop AI chat app for people who want to run models on their own computer while retaining the option to use cloud providers. Local model use works offline; choosing a hosted model sends requests to that provider.

Slotstream runs Qwen3.8-Flash-Next on Apple Silicon Macs with too little RAM to hold the whole model. It's aimed at Macs with 16 to 64 GB of memory, for people who want local chat, answers about pictures or a backend for coding assistants.

ChuanhuChatGPT is a self-hosted web interface for people who want to chat with local LLMs and cloud models in one application. It also supports document questions and automated agent tasks.

Nativ is a macOS app for people who want AI chat and coding tools to use models running on their own Mac. It combines a chat workspace, model library, and local inference server.

Ollama runs open models for people who want local AI applications, model chat or a backend for coding assistants. It provides a command-line interface and an API, with a choice between inference on your hardware and its cloud service.

LLM is a command-line tool and Python library for developers who want to use language models in terminal workflows or applications. It supports models running on your machine alongside remote providers such as OpenAI, Anthropic and Google.

exo is a local LLM runner for people who want to run models too large for a single machine's memory. It joins devices into a cluster and runs inference on your own hardware.

Icebreaker is a local AI chat app for people who want to converse with language models running on their own hardware. It uses llama.cpp, with Hugging Face also part of its model stack.

mistral.rs is an inference engine for developers who want to run AI models on their own computer or server. It supports browser chat and inference embedded directly in Python or Rust applications.

DwarfStar is a local LLM inference engine for people who want to run large DeepSeek, GLM and Qwen models on their own computers or servers. It focuses on selected models and hardware setups, and requires the project's own GGUF files rather than accepting arbitrary GGUF downloads.

Odysseus brings AI chat, autonomous agents and personal work into one workspace on your own hardware. It's for people who want models to work with their documents and ongoing tasks in a self-hosted app.

Aphelion is a desktop app for people who want character conversations, creative writing and coding help from models running on their own computer. It works offline after the first model download and doesn't require an account.

Izwi is a local voice AI app and runtime for people and teams processing speech on their own hardware or inside infrastructure they control. It brings audio tools and voice conversations into a desktop app, browser interface, CLI and inference server.

Braina combines AI chat, dictation and computer control for people who want to work through spoken or typed commands. It can run language models on your own computer and help with writing, document research and repetitive desktop tasks.

Klee is a native macOS AI chat app and agent for people who want an assistant to work with files on their Mac. It runs models locally through MLX, and downloaded models can chat offline without an account or API key.

MLX LM is a Python package for developers and researchers who want to generate text and adapt language models on Apple Silicon Macs. It uses MLX for local inference and training, with access through Python or a terminal.

AI Runner is a desktop app for people who want an AI companion and an art workspace on their own hardware. Its local chat, voice and image tools work offline without an API key or subscription.

MLC LLM is a compiler and deployment engine for developers running language models on their own hardware or building AI into apps. Its shared MLCEngine supports deployment across desktop computers, mobile devices and browsers.

DL4MicEverywhere lets microscopy researchers train and use deep learning models on their own computers through graphical Jupyter notebooks. It builds on ZeroCostDL4Mic, bringing its bioimage analysis workflows to local machines and remote computing infrastructure. You don't need coding skills to use the notebook interfaces.