Local & Self-Hosted AI Tools

ONNX Runtime is an open source inference and training engine for developers building AI into apps and services. It runs ONNX models across desktop systems, mobile devices, web browsers and servers. It's a fit when you need the same model format to work in several places, including on a user's device.

pyVideoTrans translates spoken audio into another language and produces a video with translated subtitles and AI dubbing. It's for people adapting videos for audiences in other languages who want control over which parts run locally. It recognizes speech directly, so the original video doesn't need subtitles.

TRL is a Python library for developers and researchers who want to adapt foundation models on their own hardware. It builds on Hugging Face Transformers and covers supervised fine-tuning, reinforcement learning and training from preference feedback. It's open source under Apache 2.0.

Stable Diffusion web UI (AUTOMATIC1111) is a browser interface for generating and editing images with models running on your own hardware. It's for artists and anyone who wants control over prompts, models and image variations. The software is open source under AGPL-3.0.

Kohya's GUI lets you train and fine-tune image generation models on your own GPU-equipped computer through a browser interface. It's for artists and model makers who want to teach a model a particular style or subject while controlling the training settings. The interface builds on Kohya's Stable Diffusion training scripts, with a command-line interface available too.

ThunderAI brings AI writing and email processing into Thunderbird for people who want help with their inbox while choosing where their messages go. It can use local models through Ollama or an OpenAI-compatible server such as LM Studio. Cloud connections send the selected content to ChatGPT, the OpenAI API, Google Gemini or Claude instead.

Kokoro is a text-to-speech model and inference library for developers who want to generate speech on their own hardware or servers. Its compact Kokoro-82M model suits personal projects and production applications, with Apache 2.0 licensing for both the library and model weights.

Stability Matrix is an open source desktop app for people who use more than one Stable Diffusion interface. It manages local installations of ComfyUI, Automatic1111, Fooocus, Forge, and InvokeAI, so you can try different workflows without maintaining a separate model collection for each one. It runs on Windows, macOS, and Linux under the AGPL-3.0 license.

MCP Inspector is a locally run developer tool for testing Model Context Protocol (MCP) servers. It's for developers building or checking the servers that connect AI applications to tools and data. You can inspect a local server or connect to a remote HTTP endpoint.

PR-Agent is an open source AI code review agent for development teams that want to choose where their reviewer runs and which model it uses. It reviews pull requests, writes descriptions, suggests code improvements, and answers questions about proposed changes. You can run it locally through a CLI, on a self-hosted server, in Docker, or through GitHub Actions.

Ollama JavaScript connects Node.js and browser applications to models running through Ollama. It's for developers building chat interfaces, AI agents or other apps that need a local LLM backend. The library is open source under the MIT license, with TypeScript types and an API that follows Ollama's REST interface.

ms-swift is a Python framework for developers and researchers who want to train and deploy language or multimodal models on their own hardware. It brings fine-tuning, evaluation and model serving into one project, with support for Qwen3, DeepSeek-R1, Llama4 and Mistral, plus multimodal models such as Qwen3-VL and InternVL3.5. It's open source under Apache 2.0.

Doccano is a self-hosted text annotation tool for machine learning practitioners who need labeled training or evaluation data. It runs on your own machine or server, with a browser interface and Docker support. The software is open source under the MIT license.

Stable Diffusion 3.5 is a family of text-to-image models for people building image tools or producing visual work on their own infrastructure. It generates photography, paintings, line art and 3D-style images from prompts, with an emphasis on following the requested subject and composition.

Continue is an open-source coding agent available as a CLI, VS Code extension and JetBrains plugin. Continue was acquired by Cursor, and the project is no longer actively maintained. Its repository is read-only, and its software uses the Apache 2.0 license.

Tesseract is an open source OCR engine for extracting text from images, with a command line program and a library developers can embed in their own applications. It's suited to document processing workflows and software that needs text recognition. The project uses the Apache 2.0 license and doesn't include a graphical app.

Superwhisper is an AI dictation app for macOS, Windows, iOS and Android that turns speech into text in the app you're using. It's for people who prefer speaking to typing, including developers dictating requests to coding assistants. Speech recognition can run locally and offline, or use cloud models for remote processing.

Alpaca is an open source AI chat client for people who want to run models on their own device. It uses Ollama to download and manage local models, then lets you chat without an internet connection. Conversations stay on your device in SQLite files, and local models have no direct internet access.

AI Toolkit (ostris) is an MIT-licensed training suite for people who want to fine-tune image and video models on their own hardware or a self-hosted server. It targets consumer NVIDIA GPUs and runs on Linux and Windows, including ARM64 Linux systems such as DGX Spark. An experimental installer also supports Apple Silicon Macs. GPU memory needs depend on the model and training task.

CTranslate2 is an open-source C++ and Python library for developers running Transformer models on their own hardware or servers. It handles translation, text generation, text encoding and speech recognition. Its custom runtime focuses on reducing inference time and memory use compared with general-purpose deep learning frameworks.

Molmo is Ai2's family of vision-language models, with code for running and training models on your own hardware. It's for developers and researchers who need to work with images and text, adapt a model, or evaluate it against visual tasks. The Python codebase is open source under Apache 2.0 and builds on OLMo, adding image encoding and generative evaluation.

Argos Translate runs neural machine translation locally, so you can translate text without sending it to an online service. It's for people who need offline translation and developers who want to add it to their own software. The Python library and desktop interface use the same translation engine, and the project is open source under the MIT license.

WhisperX is an open source speech-to-text tool for people transcribing interviews, meetings, and long recordings on their own computer. It builds on OpenAI's Whisper to produce transcripts with word-level timestamps and optional speaker labels.

Qwen Code is an Apache 2.0 licensed AI coding agent for developers who want help working through a codebase, changing code and checking the result. It runs on macOS, Windows and Linux, with a terminal interface and a desktop app. It builds on Google Gemini CLI and has developed into an agent that can use Qwen models alongside other model providers.

Typebot is a visual chatbot builder you can host on your own server or use through its managed cloud service. It's for businesses building conversational forms and chatbots for websites, mobile apps, and WhatsApp, with developers able to extend flows through APIs and JavaScript. The project uses Fair Source licensing.

Google ADK is an open-source AI agent framework for developers building applications that carry out multi-step tasks. You can run agents locally in Docker or on your own infrastructure, and connect them to locally running models through adapters. The framework is optimized for Gemini but supports other models and providers. Its Python repository uses the Apache 2.0 license.

SWE-agent is an open-source AI coding agent that lets a language model use tools to attempt fixes for issues in GitHub repositories. It's aimed at developers and researchers studying how agents handle real software tasks. The project is in maintenance-only mode: mini-swe-agent has superseded it, and the maintainers recommend that successor for new users.

OLMo is a family of language models for researchers and developers who want to inspect, adapt or run a model on their own hardware. Weights are downloadable. Ai2 also provides the training data, code, checkpoints and reports behind the models, giving researchers material to study the full training process rather than only the finished model.

Wren AI is a self-hosted data agent for teams and agent builders who need answers based on agreed business definitions. It turns plain-language questions into SQL and interactive dashboards, using the same definitions when someone asks directly or through Claude, ChatGPT or Gemini over MCP.

Chroma DB is an open-source search database for developers building AI apps and agents that need to retrieve information from their own data. You can run it locally or host it on your own infrastructure under the Apache 2.0 license. Chroma Cloud is a separate hosted service for managed, serverless search.

kotaemon is a self-hosted document chat app for people who want to ask questions across their files and check where the answers came from. It runs in a browser on Windows, macOS or Linux, with Docker also supported. The project uses the Apache 2.0 license.

llama-cpp-python brings llama.cpp model inference into Python applications and exposes it through a self-hosted OpenAI-compatible server. It's for developers building local AI applications or connecting existing API clients to models on their own hardware. The package is open source under the MIT license.

RamaLama runs and serves AI models on your own hardware using OCI containers. It's aimed at developers who want local chat or a self-hosted inference API with a container workflow they can also use in production. The project uses the MIT license.

KoboldCpp pairs local model inference with a browser interface built for chat, creative writing and roleplay. A fork of llama.cpp, it bundles KoboldAI Lite with tools for keeping character details and story context alongside your conversations. It's open source under AGPL-3.0.

Skyvern uses vision models to carry out tasks in a browser, such as filling forms, collecting data, and working through logins. It's for teams automating websites whose layouts change, as well as developers who want AI actions alongside Playwright. Instead of depending on fixed page selectors, it identifies visible elements and decides how to interact with them.

Florence-2 is Microsoft's open-source vision model for developers who want to process images on their own hardware. It handles several image tasks through text prompts, so one model can generate descriptions, read text and locate objects. It runs locally with PyTorch and Hugging Face Transformers on a CPU or CUDA GPU, and uses the MIT license.

Transformers.js is a JavaScript library for developers building web apps that run AI models on the user's device. Inference happens in the browser, so an app doesn't need a separate model server to process its inputs. The library is open source under Apache 2.0.

aichat brings Ollama and cloud AI services into the same terminal interface for developers and people who work at the command line. It runs locally on macOS, Linux and Windows, with Android support through Termux. Model processing happens through the backend you choose: Ollama supports local models, while providers such as OpenAI, Claude and Gemini process requests in the cloud.

Aider is an Apache 2.0 open-source coding assistant for developers who work in a terminal and keep their projects in Git. It can help start a project or make changes to an existing codebase. You can use local LLMs or connect to cloud models from providers including Anthropic, DeepSeek and OpenAI.

Guardrails AI is an open-source Python framework for developers who need to check what goes into an LLM and what comes back. It runs within your application or as a self-hosted service. The framework uses the Apache 2.0 license and helps address risks such as policy violations, hallucinations, and data leakage before outputs reach users.

FramePack is an open source desktop app for making videos from a still image and a written motion prompt. It runs on Windows and Linux, with generation handled by your own NVIDIA GPU. It suits people who want to make AI video locally and see the clip develop as it renders.

Wyoming Faster Whisper is a local speech-to-text server for Home Assistant and other clients that use the Wyoming protocol. It turns spoken audio into text on your own hardware, with support for names specific to your home. It's open source under the MIT license and runs as a Home Assistant add-on, a Docker container, or a local Python service.

Semantic Kernel is an MIT-licensed SDK for developers adding AI agents to their applications. It supports local models through Ollama, LMStudio and ONNX, alongside cloud services such as OpenAI and Azure OpenAI. You choose the model backend. The SDK runs on Windows, macOS and Linux and supports C#, Python and Java.

verl is a Python library for teams training large language models on their own GPU infrastructure. It's the open-source implementation of HybridFlow, aimed at researchers and engineers who need reinforcement learning after initial model training. It uses the Apache 2.0 license.

Typesense combines typo-tolerant site search with vector and semantic search in a self-hosted engine. It's for developers building searchable apps, product catalogs or AI search over their own data. The C++ engine uses an in-memory architecture for low-latency results as users type.

python-audio-separator separates recordings into vocals, instrumentals and individual instruments on your own hardware. It's an open-source Python package under the MIT license, aimed at karaoke creators and developers who want audio separation in scripts or their own applications.

PocketPal AI is an open source assistant for people who want to run language models on a phone or tablet. It works on iOS, iPadOS and Android. Once you've downloaded a model, you can chat offline without an account, and your prompts, replies and documents stay on your device. The app is licensed under MIT.

Buzz transcribes and translates speech on your own computer using OpenAI's Whisper. It's for people who need transcripts or subtitles from recordings, plus live captions from a microphone. Local transcription works offline; the optional OpenAI Whisper API sends audio to a cloud service.