Local & Self-Hosted AI Tools

AutoGPT is an AI agent platform for teams handling recurring business work, with a hosted service and a self-hosted option. Its coordinator, Otto, assigns work to specialists for content, sales, research and operations. You can describe an outcome in plain English or talk directly to the specialist responsible for it.

AutoGen is a framework for developers building AI agents that work together or alongside people. Its agent runtime can run locally or across distributed systems, while model integrations such as OpenAI and Azure OpenAI send requests to external services. It's community-managed and in maintenance mode, with no further features or enhancements planned.

SuperAGI is a self-hosted AI agent framework for developers who want to build task automation and manage agents through a browser interface. It runs locally in Docker and can use local LLMs with a GPU. The Python project uses the MIT license, so you can modify it for your own applications.

OGX, formerly Llama Stack, is a self-hosted AI application server for developers building chat apps, document search or AI agents. It brings model inference, file storage, vector search and agent orchestration into one process. You can run it on a laptop, in a datacenter or in the cloud. It's open source under MIT.

Serge is a self-hosted chat interface for people who want to run language models on their own hardware and talk to them in a browser. The project is archived and no longer maintained. It uses llama.cpp to run models locally, with Alpaca as a named chat model and LLaMA also referenced in its memory requirements.

LlamaGPT is a self-hosted ChatGPT alternative for people who want general chat or coding help on their own computer or home server. It runs models locally and keeps conversation data on your device. After the initial model download, it works offline.

Layla is an offline AI assistant for Android and iOS that runs language models on your phone. It's for people who want private everyday chat or an AI companion with custom characters, memory and roleplay. Local conversations stay encrypted on your device; optional cloud mode sends requests to your chosen hosted provider.

Lollms WebUI is a local, single-user AI interface for people who want text chat and media generation in one place. It runs on Windows, macOS and Linux, with Docker support, and lets writers, developers and other users choose models and task-specific personalities. It's free and open source under Apache 2.0. The project receives minimal maintenance.

Chatbot UI is a browser-based AI chat app for people who want to host their own interface and choose between local models and cloud providers. It connects to Ollama for models running on your hardware, alongside services such as OpenAI, Azure OpenAI, Anthropic and Google Gemini. The app is open source under the MIT license.

Sonar is a self-hosted inference engine for developers and teams serving Hugging Face-compatible language and multimodal models on their own hardware. Based on vLLM, it adds model and quantization formats, sampling methods, and deployment features. It's open source under AGPL-3.0.

Stable Diffusion XL is an image generation model for creators and developers who want to generate images on their own hardware or servers. It supports text-to-image generation and image-to-image sampling, so you can start with a written prompt or an existing image.

Qwen-Image is an open-source image generation and editing model you can deploy locally. It's for developers and creators who want to generate images from text or revise existing pictures on their own hardware. Its text rendering capabilities, especially for Chinese, make it relevant for images that need readable lettering alongside visual content.

Stable Diffusion 1.5 is an AI image generation model for creators and developers who want to generate images on their own hardware. It turns text prompts into images and supports text-guided changes to existing pictures, including turning rough sketches into detailed artwork. Local inference keeps that image generation work on your machine.

Nexa SDK is an on-device AI inference framework for developers building applications that process text, images or audio on users' hardware. It runs models locally across CPUs, GPUs and NPUs, with a shared interface for different backends. Its scope includes language and vision models, speech recognition, speech synthesis and image generation.

Qwen2.5-VL is a vision-language model you can run on your own hardware to answer questions about images and video. It's aimed at developers building document processing tools, visual assistants and agents that interact with computer or phone screens. The instruction-tuned 7B model has Apache 2.0 licensing and works with Hugging Face Transformers, with weights available in Safetensors format.

Pixtral is a family of Mistral models for developers who want to run multimodal AI on their own hardware. The associated mistral-inference project is archived and no longer maintained. That status applies to the inference library.

SmolVLM is a compact vision language model from Hugging Face for developers building local AI applications that work with images and text. It can describe pictures, answer questions about diagrams, and read information from documents such as invoices. Its small memory footprint makes on-device use practical on laptops and smaller local setups.

MNN is a lightweight C++ framework for developers who want AI models to run on phones, PCs and embedded devices. It handles inference and training on the device, with a focus on small application footprints and hardware acceleration. The project is open source under Apache 2.0, and Alibaba uses it in apps including Taobao, Youku and DingTalk.

DB-GPT is a self-hosted AI data assistant for teams analyzing business data and developers building data applications. It turns plain-language requests into SQL queries and Python analysis, then produces charts, dashboards, or HTML reports. You can run it on macOS or Linux, with Docker deployment also supported.

BotSharp is a self-hosted framework for .NET developers building AI agents into business applications. Written in C#, it runs on Windows, Linux and macOS and is open source software under Apache 2.0. Its plugin design lets teams choose their model provider, storage and interface while keeping agent coordination in the same framework.

LibrePhotos is a self-hosted photo manager for people who want to organize a photo library on their own server, with AI tools for finding images and grouping faces. It scans files already on your storage and supports RAW photos as well as videos. The project is open source under the MIT license.

Qwen2.5-Coder is a family of code-focused language models for developers who want to run a coding assistant on their own hardware or build a self-hosted code agent. It covers code generation, reasoning about code and fixing errors. The 32B instruction-tuned model is open source under Apache 2.0 and accepts requests in a chat format.

ZoneMinder is a self-hosted video surveillance system for people who want to manage security cameras on their own Linux machine. It brings video capture, analysis, recording and monitoring into one system, with support for IP, USB and analog cameras. It's free and open source under the GNU General Public License v2.0 (GPL-2.0).

Jamba is AI21's language model family for teams building AI applications on their own servers. The documented Large 1.7 model combines Mamba state-space models with Transformer attention to process long context efficiently. A 256K-token context window makes it relevant for work that depends on lengthy documents, such as investment research, due diligence and reviewing procurement responses.

Kaito manages self-hosted LLM inference, fine-tuning, and document retrieval services in a Kubernetes cluster. It's for teams that want to run models on infrastructure they control while reducing the work of sizing GPU resources and managing model deployments. The project is open source under Apache 2.0.

Codestral and Devstral are Mistral coding models with downloadable weights for local or self-hosted development tools. Codestral focuses on code generation and fill-in-the-middle completion, where the model fills a gap between existing code. Devstral is designed for software engineering agents that explore a repository, use tools and edit multiple files.

Command A is an open weights language model from Cohere and Cohere Labs for researchers and developers building self-hosted chatbots, document assistants and AI agents. Its focus is business tasks that combine multilingual text, supplied documents and external tools. You can run the model on your own hardware; Cohere also offers hosted chat through a playground and Hugging Face Space.

Seldon Core 2 is an AI model serving framework for teams running production machine learning and LLM applications on Kubernetes. It can run on your own infrastructure or in a cloud environment. Its focus is managing individual models and connected applications within the same deployment system.

Nemotron is NVIDIA's family of AI models for developers building agents that reason, write code and call tools. You can run models locally for private, offline work or deploy them on your own servers. NVIDIA publishes model weights, training data and recipes so teams can inspect and adapt the models for their applications.

Mistral Small and Large are downloadable language models for developers building chat, reasoning and tool-using applications on their own infrastructure. Capabilities and hardware requirements depend on the release. Mistral Small 3.1 adds image understanding to text generation, while Mistral Large 2 is a larger text model.

GLM-4.5 is an open-source language model for developers building AI agents and coding tools on their own servers. It combines reasoning with tool calling and offers a choice between thinking mode for complex tasks and non-thinking mode for direct responses. The MIT license permits commercial use and modification.

LangChainGo is a Go implementation of LangChain for developers building LLM applications in their own software. It connects Go programs to model backends, including Ollama for local LLM use and cloud services such as OpenAI and Gemini. It's a library, so its audience is developers who want to build an application rather than use a ready-made chat interface.

lm-evaluation-harness lets researchers and model developers compare language models using shared academic benchmarks and public prompts. It runs evaluations against local models and benchmarks, or sends requests to a hosted model API. EleutherAI's Python framework is open source under the MIT license and powers Hugging Face's Open LLM Leaderboard.

Plano, formerly Arch Gateway, is a self-hosted AI gateway for developers building applications with multiple agents or model providers. It puts routing, guardrails and request tracing in a separate service, so each agent doesn't need its own implementation of that infrastructure. It's open source under Apache 2.0.

Hallo turns a single portrait and a speech recording into an animated talking video on your own hardware. It's a local AI tool for creators working with talking portraits and researchers who want access to both generation and training code. The Python code uses the MIT license; required pretrained models and dependencies have their own terms.

SwarmUI is a self-hosted AI image generation interface that combines a straightforward generation screen with direct access to ComfyUI's node-based workflows. It's for people who want to run creative models on their own hardware, with room to build more detailed workflows as their needs grow.

Unstructured is a local document processing library for developers building LLM applications and document ingestion pipelines. It turns PDFs, Word documents, HTML, emails and images into document elements that applications can use. The Python library is open source under Apache 2.0 and runs on your own hardware, including through Docker images for x86_64 and Apple Silicon.

Portkey Gateway is a self-hosted AI gateway for developers whose apps need to use local models and cloud providers through one OpenAI-compatible API. It routes requests to Ollama, OpenAI, Anthropic, Google Gemini and other backends, with controls for handling failures and checking model inputs and outputs.

Morphik Core is a self-hosted multimodal retrieval engine for developers building AI applications around visually rich documents. It searches diagrams, schematics, charts, and datasheets alongside text, so applications can retrieve information that text extraction alone can miss. You can run it on your own server, including through Docker, or use Morphik's hosted service.

Flowframes is a Windows desktop app that uses local AI to generate intermediate frames for smoother video and animation. It's for people working with camera footage, 2D animation or rendered video who want a graphical interface for interpolation. The app is open source under GPL-3.0 and available as donationware, with free builds.

Marqo Open Source is a self-hosted search engine for developers building semantic document search, image search, or retrieval for AI applications. It handles embedding generation alongside storage and retrieval, so applications can submit documents without maintaining a separate embedding service. The open-source project is deprecated and no longer receives updates.

USearch is an open-source similarity search library for developers building semantic search, recommendation systems, and other applications that compare vectors. It runs within your application on your own hardware or server. Its compact C++ core supports custom definitions of similarity, including comparisons between combined image and text embeddings or geospatial data.

Amuse combines AI generation with media editing in a Windows app that runs models on your own hardware. It's for people who want to create images, video, audio and text locally, then work on the results in the same application. Its editor also accepts existing local video.

Deforum is a free Stable Diffusion tool for artists and developers who want to generate images and animations on their own hardware. The project is no longer maintained. Its code uses MIT, while downloaded Stable Diffusion models retain their own weight licenses. Its Python script and Jupyter notebook support local generation, while Google Colab and Replicate offer cloud execution.

Plandex is an open source, terminal-based AI coding agent for developers working on changes that span many files. Its review sandbox holds generated edits apart from your project files so you can inspect the accumulated changes before applying them. You can run its server locally through Docker or host it on your own server. The code uses the MIT license.

NextChat is a self-hosted AI chat interface for people who want one place to use their own LLM server and cloud models. The web and desktop project is open source under the MIT license. You can host it with Docker or on Vercel, and desktop clients run on macOS, Windows and Linux.

LLM Farm runs large language models offline on iOS and macOS. It's for people who want on-device AI chat or need to compare how different models perform on Apple hardware before choosing one for a project. The app is open source under the MIT license, with ggml and llama.cpp handling local inference.

R2R is a self-hosted AI retrieval system for developers building applications that answer questions using their own documents. It combines search, retrieval-augmented generation (RAG) and a reasoning agent behind a REST API. The project is open source under the MIT license and runs as a Python service or in Docker.