Self-Hosted AI Tools You Can Run in Docker

Verba is a self-hosted document chatbot for people who want to ask questions across their files and knowledge bases. The project is archived and no longer maintained, so it won't receive bug fixes or security patches.

LlamaIndex Chat is a chatbot builder for people who want assistants that answer questions about their own documents. It's an example application built with LlamaIndexTS; you can host the app yourself, but it relies on cloud services for document indexing and AI responses.

MosaicML Composer is a Python library for training neural networks on your own GPUs or multi-node clusters. Built on PyTorch, it's aimed at researchers and engineers who want distributed training with room to customize their experiments. It's open source under Apache 2.0, and prebuilt Docker images are available.

Codestral and Devstral are coding models for developers who want to run completion tools or software engineering agents on their own hardware. The associated mistral-inference library is archived and no longer maintained.

Flowise is a visual builder for developers and teams creating AI agents and chatbots from connected workflow blocks. The project is archived and no longer maintained.

pgai is a Python library for developers building RAG applications and AI agents around PostgreSQL data. The project is archived and no longer maintained or supported.

TensorZero is a self-hosted platform for developers who need to compare and improve models in LLM applications. The project is archived and no longer maintained.

ollama-telegram is a Telegram bot for people who want to chat with an Ollama model through their messaging app. The project is archived and no longer maintained.

Clawlet is a self-hosted personal AI agent for people who want an assistant on their own machine or server. It supports local models and can retrieve past context through optional memory search.

NVIDIA RAG Blueprint is a reference framework for developers building AI question answering over enterprise data. It retrieves relevant material from your documents to ground answers and can cite the retrieved sources.

Cognita is a self-hosted RAG framework for developers building applications that answer questions from their own documents. The project is archived and no longer maintained.

Intel IPEX-LLM is a library for developers running or fine-tuning language models on Intel hardware. The project is archived and no longer maintained, and Intel has identified known security issues in the code.

TaskWeaver is an AI agent framework for developers building assistants that analyze data through Python code. The project is archived and no longer maintained.

MetaMCP is a self-hosted MCP gateway for developers and teams connecting AI clients to multiple tool servers. It lets you manage the tool collections those clients receive through shared endpoints.

Pixtral is a family of Mistral models for developers building multimodal applications with downloaded weights. The associated mistral-inference library is archived and no longer maintained.

Mistral Small and Large are downloadable language models for developers building chat and text applications on their own infrastructure. Their reference library, mistral-inference, is archived and no longer maintained.