Self-Hosted AI Tools You Can Run in Docker

OpenLIT is a self-hosted platform for developers who need to understand how their LLM applications and AI agents behave. It connects model calls with tool activity, retrieval and agent steps, so teams can investigate errors and compare cost, latency and output quality across a workflow.

Paperless-GPT adds AI text extraction and document organization to an existing paperless-ngx library. It runs in Docker on your own server and suits people who want less manual sorting of scanned paperwork. The project is open source under the MIT license.

Higress is a self-hosted AI gateway for developers and teams managing model APIs and the tools their AI agents call. It puts LLM traffic and MCP servers behind a shared entry point, with authentication, traffic controls and monitoring. The open-source edition uses the Apache 2.0 license and runs locally in Docker without registration. Alibaba Cloud also offers a fully managed gateway.

Infinity is a self-hosted database for developers building search and retrieval-augmented generation (RAG) into LLM applications. It combines embedding search with full-text search and structured filters, so an application can retrieve relevant records through both meaning and exact terms.

CompreFace is a self-hosted face recognition service for developers who want to add facial identification to an application without building or training their own machine learning system. It runs as a Docker-based server on your hardware or in a cloud deployment you manage. It's free and open source under the Apache 2.0 license.

SimpleTuner is an open-source toolkit for fine-tuning image, video and audio generation models on your own hardware or GPU servers. It's for creators and researchers adapting models to their datasets, and teams sharing training infrastructure. A web dashboard manages training jobs.

MetaMCP is a self-hosted gateway for developers and teams who want to give AI clients access to several MCP servers through one endpoint. It runs on your own machine or server with Docker and is open source under the MIT license. You choose which tools clients see.

XTuner is an open-source LLM training engine for researchers and teams training large mixture-of-experts (MoE) models on their own hardware. It supports GPU and Ascend NPU training, with an emphasis on memory use and distributed training efficiency at scales reaching a trillion parameters.

Oumi builds specialized AI models for teams that want control over their training data, model weights, and deployment. Its Apache 2.0 open-source stack runs on laptops, clusters, and your own servers, while its hosted service automates model development from a plain-English task description. You own the resulting weights, data, and training recipes.

OpenLLM is a self-hosted LLM server for developers who want to connect their applications to models running on their own hardware or servers. Its OpenAI-compatible API works with clients built for that interface, including the OpenAI Python client and LlamaIndex. The project is open source under the Apache License 2.0.

VideoLingo is a self-hosted video translation app for creators and educators who need bilingual subtitles or dubbed versions of their videos. It brings transcription, translation and subtitle timing into one browser interface, with dubbing as an optional output. The project is open source under Apache 2.0; a separate hosted service offers subtitle translation and dubbing.

Harbor is a CLI and companion app for people experimenting with AI on their own hardware. It manages a local LLM development environment, connecting model backends to chat interfaces and supporting services so you don't have to configure each connection yourself. It's open source under Apache 2.0.

AllTalk TTS generates speech on your own computer. The project recommends v2 for most users; the saved documentation below describes v1, built on Coqui TTS and XTTSv2 models. It's for people adding voices to AI conversations or producing spoken audio from longer texts. It runs as a standalone application or alongside Text-generation-webui, with support for Windows, Linux and macOS.

ModelScope combines a hosted model and dataset hub with a Python library you can run locally. It's for developers and researchers who want to use AI models in their own applications, fine-tune them on their own data, or compare their performance. The library is open source under Apache 2.0.

Claude Code Router is an open-source local model gateway for developers who use coding agents and want to manage their model providers in one place. It runs on macOS, Windows and Linux, with Docker and a CLI with a browser interface also available. The project uses the MIT license.

Scriberr is a free, open source transcription app for people who want to keep meeting recordings and voice notes on their own hardware. It turns audio and video into text locally, with offline transcription once the models are downloaded. The MIT license allows you to use and modify it.

Wyoming Piper connects Piper's local text-to-speech engine to Home Assistant and other clients that use the Wyoming protocol. It's for people building a voice assistant on their own hardware who need speech generation as a self-hosted service. The project is open source under the MIT license.

NeMo Curator is an open source Python toolkit for ML engineers and data teams preparing AI training datasets on their own hardware. It handles text, images, video and audio, with reusable pipelines that can run on a laptop or scale across a multi-node Ray cluster. NVIDIA uses it to prepare data for Nemotron models.

Chat2DB Community is a database client and AI SQL workspace for developers, database administrators and analysts. It runs on Windows, macOS and Linux, with Docker and browser access also available. Its AI assistant connects to a model you configure to generate, explain and optimize SQL. Where AI requests are processed depends on that model connection.

ClearML is an MLOps suite for recording experiments, managing datasets and running ML workloads. Its Apache 2.0 Python SDK connects to a ClearML Server, available as a hosted service or open-source software you deploy yourself. ClearML Agent handles job orchestration and reproducibility.

ChatDev is a self-hosted platform for building teams of AI agents through a visual workflow editor. It's for people who want agents to collaborate on research, data analysis or software projects without writing the orchestration code themselves. You define each agent's role and how information passes between them.

TTS WebUI brings local text-to-speech, music generation and audio processing into one browser interface. It's for people creating spoken audio or music, and for developers who want to add speech to a self-hosted chat app. The interface combines Gradio and React, with extensions that let you choose which audio models to use.

HomeGallery is a self-hosted web gallery for people who want to browse their personal photos and videos while keeping their existing files and folders. It brings multiple media directories into one gallery, with a mobile-friendly browser interface and PWA support. It's open source under MIT.

dstack is a self-hosted orchestration tool for AI teams managing compute across GPU clouds and their own servers. It puts cluster management, training jobs and model inference behind one interface, so teams can use different providers and accelerators without maintaining a separate workflow for each environment. It's open source under the Mozilla Public License 2.0.

jetson-containers is a Docker container build system for developers running local AI and robotics workloads on NVIDIA Jetson hardware. It supplies prebuilt images and lets you combine AI packages into custom containers, reducing the work of assembling compatible GPU software for JetPack/L4T.

Kodus is an AI code review tool for engineering teams that want automated pull request feedback while choosing where the reviewer runs and which model it uses. Its open source core uses the AGPL license and can be self-hosted with Docker Compose or on Kubernetes. Kodus also offers a hosted cloud service that manages the infrastructure.

llmcord is a self-hosted Discord bot for people who want to share LLM conversations with friends or a community. You can run the Python bot on your own machine or server, including through Docker, and connect it to local models or cloud providers. It's open source under the MIT license.

GPT-SoVITS is a local text-to-speech and voice cloning tool. It can generate speech from a short reference recording or fine-tune a model for a custom voice. The source code uses the MIT license.

Postgres MCP Pro is a self-hosted MCP server that gives AI assistants access to PostgreSQL alongside tools for diagnosing slow queries and testing index recommendations. It's for developers and database maintainers who want their assistant to work with the database's actual schema, query statistics and execution plans. It runs through Docker or Python and is open source under the MIT license.

Voice Changer (w-okada), also called VCClient, converts your voice as you speak using AI voice models. It's for people who want live voice conversion on their own computer, including those recording gaming commentary while running demanding software. Processing can stay local.

OpenVoiceOS is a free, open-source voice AI platform for people building their own smart speakers or adding voice control to devices. Its core builds on a fork of MycroftAI/mycroft-core, and most classic Mycroft skills also work with it. The project uses the Apache 2.0 license, which permits personal and commercial use.

GitHub MCP Server is GitHub's official connector for developers who want their AI assistant to work with repositories and project activity. It gives compatible coding assistants access to GitHub through the Model Context Protocol (MCP), so they can retrieve context and take actions requested in natural language. The code is open source under the MIT license.

MetaGPT organizes AI agents into a software development team, with roles for product management, architecture, project management and engineering. It's a self-hosted Python framework for developers who want to generate software projects from plain-language requirements or build their own collaborative agents.

Petals lets developers and researchers use large language models that won't fit on a single consumer GPU by sharing the work across a network of machines. It supports text generation and fine-tuning from a desktop computer or Google Colab. Each participant holds part of the model, while other computers handle the remaining parts.

DeepFace is a Python library for developers building face recognition into their own applications. It runs locally or as a self-hosted API, including through Docker. A separate managed API at deepface.dev handles processing on hosted infrastructure. The library is open source under the MIT license.

Parakeet is NVIDIA's speech recognition model family. The linked parakeet-tdt-0.6b-v2 is its English speech-to-text model for developers and researchers building transcription services, subtitles or voice applications. It runs locally through NeMo on Linux, with NVIDIA GPUs recommended for inference. It's a model you can embed in an application, rather than a desktop transcription app.