Self-Hosted AI Tools You Can Run in Docker

Civitai is a self-hostable platform for sharing AI image models and artwork. Its Apache-2.0 code provides accounts, model uploads, browsing and comments. The public Civitai website also operates a hosted image generator, which is separate from the runnable local platform.

gpt-engineer is a locally run coding assistant for developers who want to experiment with AI code generation and build their own agents. It can create software from plain-language descriptions or make requested changes to an existing codebase. The project is archived and no longer maintained. Its Python code is open source under the MIT license.

Devika is a self-hosted AI coding agent for developers who want to give a software task in plain language and have an agent plan the work, research it and write code. Modeled after Cognition AI's Devin, it runs on your own machine with a browser interface and supports local LLMs through Ollama. It's open source under the MIT license.

InstantMesh generates a 3D mesh from a single image on your own machine. It's for creators exploring image-based 3D assets and researchers working on 3D reconstruction. The Python project is open source under Apache 2.0 and uses PyTorch with CUDA for GPU processing.

Nomic Embed Text v1.5 is an English text embedding model for developers building semantic search, document retrieval, and RAG applications on their own hardware or servers. It turns text into numerical representations that applications can compare by meaning. Its main distinction is adjustable embedding size: you can use smaller vectors when storage matters, with a tradeoff in retrieval quality.

Aim is a free, open source ML experiment tracker for researchers and teams who want to keep training records on their own infrastructure. It runs in your training environment or on a self-hosted server, with Docker and Kubernetes deployment support. Its Apache 2.0 license permits use and modification.

DiffRhythm is a local AI music generation model for musicians, developers and researchers who want to create full-length songs on their own hardware. It uses latent diffusion to generate songs with vocals and accompaniment, and can also produce instrumental music. The full model supports songs up to 4 minutes and 45 seconds.

Twinny is an AI coding assistant for VS Code that lets developers choose where their models run: on their own computer, a private server or a hosted API. It's for individuals and teams who want code suggestions and repository chat with control over where their code goes. The extension and team gateway are open source under the MIT license.

Trilium Notes is a local-first note-taking app for people building a large personal knowledge base. It runs on Windows, macOS and Linux, or on your own server through Docker, with browser access and a mobile web interface. It's free and open source under AGPL-3.0.

Jina Embeddings is a family of models that converts content into vectors for retrieval, similarity matching, classification and clustering. It includes multilingual text models and multimodal variants for searching across different media.

ollama-telegram connects Telegram chats to a local LLM through Ollama. The project is archived and no longer maintained. It's for people who want to access their own model through Telegram, with the bot and model backend running on hardware or a server they control.

Matrix ChatGPT Bot connects Matrix rooms to OpenAI's ChatGPT API for people who want AI conversations in their existing chat client, including Element. The project is archived and no longer maintained. It's open source under AGPL-3.0, and the maintainers point users to Baibot as an alternative.

TensorRT Model Optimizer, called NVIDIA Model Optimizer or ModelOpt, is a Python library for developers preparing models for local or self-hosted inference. It reduces model size and memory use and can speed up inference through compression and other optimization techniques. It's open source under Apache 2.0.

Ludwig is an open-source Python framework for developers and researchers who want to train custom AI models on their own hardware. A YAML file describes the model and training pipeline, while Ludwig handles preprocessing, training and evaluation. It uses the Apache 2.0 license. Install the Python package with the optional LLM dependencies for fine-tuning; current source requires Python 3.12 or later.

Intel Neural Compressor is a Python library for developers compressing AI models for deployment on their own hardware or servers. It supports local LLM work as well as other deep learning models, with particular attention to Intel CPUs, GPUs and Gaudi accelerators. It's open source under the Apache 2.0 license.

H2O LLM Studio is a self-hosted tool for teams that want to adapt language models to their own datasets without writing training code. Its browser interface brings training experiments, evaluation, and model testing into one place. The project is open source under Apache 2.0.

Zonos is an open-source text-to-speech model for people who want to generate speech and clone voices on their own hardware. It can match a speaker from a short reference recording, with controls for delivery and emotion. The code uses the Apache 2.0 license.

Amica is a locally runnable interface for talking with customizable 3D AI characters. It's for people who want an animated, voiced character as the face of their AI assistant, with a choice of local LLM backends or cloud services. The project builds on Pixiv's ChatVRM.

UpTrain is an open source LLM evaluation tool for developers who need to measure answer quality and investigate failures in their AI applications. Its self-hosted web dashboard runs on your machine through Docker, with a Python package for evaluations inside application code. The dashboard requires no coding.

Langtrace is an open-source observability tool for developers who need to debug LLM applications and track their performance. You can run it locally or on your own servers with Docker and Docker Compose. Its traces follow OpenTelemetry standards.

Paperless-AI is a self-hosted extension for Paperless-ngx users who want automatic document sorting and chat with their archive. It requires an existing Paperless-ngx instance and runs in Docker, with a browser interface for reviewing and processing documents. The project is no longer maintained.

XTTS v2 generates speech from text using a reference voice recording or a preset speaker. It runs locally through Coqui TTS and suits developers building speech into apps, as well as researchers who want to fine-tune a speech model on their own hardware.

ShellGPT is an AI terminal assistant for developers and people who work with shell commands. It runs on Linux, macOS and Windows, turning plain-language requests into commands suited to your operating system and shell. You can review, explain or execute its suggestions, and its Bash and Zsh integrations put generated commands into the terminal input line for editing.

Agnai is a self-hosted AI roleplay chat app for people who want to create fictional characters and talk with them alone or in a group. A conversation can include multiple people and multiple bots. It builds on early work from Galatea-UI by PygmalionAI and uses the AGPL-3.0 open-source license.

RisuAI is an open-source AI roleplay client for people who want to create characters, build fictional worlds and chat with several characters together. It runs on Windows, macOS, Linux, Android and in a browser. You can also host the web app yourself with Docker.

Lunary is a self-hosted LLM observability and prompt management platform for teams building chatbots and AI agents. It brings production traces, user conversations and prompt versions into one place so developers can investigate errors and teams can assess response quality. You can host it in your own infrastructure or use its cloud service.

TensorZero is a self-hosted platform for developers building LLM applications. The project is archived and no longer maintained. It combines a model gateway with tools for inspecting responses, evaluating workflows, and improving prompts using production data and human feedback.

pgai keeps search embeddings in sync with PostgreSQL data for developers building RAG applications and AI agents. It's a Python library with database components and workers you can self-host, including in Docker. The project is archived and no longer maintained or supported. Its code is open source under the PostgreSQL License.

Cognita is a self-hosted RAG framework for developers building applications that answer questions using their own documents. The project is archived and no longer maintained. It combines a browser interface for document Q&A with reusable components built on LangChain and LlamaIndex, under the Apache 2.0 open-source license.

AutoGPT is an AI agent platform for teams handling recurring business work, with a hosted service and a self-hosted option. Its coordinator, Otto, assigns work to specialists for content, sales, research and operations. You can describe an outcome in plain English or talk directly to the specialist responsible for it.

SuperAGI is a self-hosted AI agent framework for developers who want to build task automation and manage agents through a browser interface. It runs locally in Docker and can use local LLMs with a GPU. The Python project uses the MIT license, so you can modify it for your own applications.

Serge is a self-hosted chat interface for people who want to run language models on their own hardware and talk to them in a browser. The project is archived and no longer maintained. It uses llama.cpp to run models locally, with Alpaca as a named chat model and LLaMA also referenced in its memory requirements.

LlamaGPT is a self-hosted ChatGPT alternative for people who want general chat or coding help on their own computer or home server. It runs models locally and keeps conversation data on your device. After the initial model download, it works offline.

Lollms WebUI is a local, single-user AI interface for people who want text chat and media generation in one place. It runs on Windows, macOS and Linux, with Docker support, and lets writers, developers and other users choose models and task-specific personalities. It's free and open source under Apache 2.0. The project receives minimal maintenance.

Sonar is a self-hosted inference engine for developers and teams serving Hugging Face-compatible language and multimodal models on their own hardware. Based on vLLM, it adds model and quantization formats, sampling methods, and deployment features. It's open source under AGPL-3.0.

Pixtral is a family of Mistral models for developers who want to run multimodal AI on their own hardware. The associated mistral-inference project is archived and no longer maintained. That status applies to the inference library.