Local & Self-Hosted AI Tools

MonkeyOCR is a local AI document parser for developers and researchers working with English and Chinese PDFs or images. It extracts text, formulas and tables while identifying page structure and relationships between blocks. That makes it useful for documents where plain text extraction loses reading order or separates content from its layout.

MMDetection is a Python toolkit for researchers and developers building object detection and image segmentation systems. Part of OpenMMLab, it combines ready-made model architectures with interchangeable components for custom models. It's open source under Apache 2.0.

BoltAI is a native Mac app for people who want local LLMs and cloud AI services in the same workspace. It runs on Intel and Apple Silicon Macs and suits coding, writing, research and document analysis. The interface uses SwiftUI and AppKit.

GPTLocalhost connects Microsoft Word to your choice of local or cloud language models. It keeps the AI interface inside Word rather than requiring a separate chat app. The provider directs teamwork use to its separate LocPilot for Word product.

LLPhant is an MIT-licensed PHP framework for adding language models, embeddings and vector databases to Symfony and Laravel applications. It requires PHP 8.1 or later and is installed through Composer.

Moshi is a voice AI model and dialogue framework that can listen while it speaks. It processes speech directly, retaining information such as emotion and non-verbal cues that a text transcription can miss. It's aimed at researchers and developers building spoken AI applications, with local inference and self-hosted server options.

torchtitan is an open-source training platform for researchers and developers building generative AI models on their own GPU machines or server clusters. It uses PyTorch's distributed training tools and keeps the model code relatively simple when spreading work across GPUs. The Python codebase has extension points and replaceable components for experiments with model architectures and training infrastructure.

nerd-dictation is an open-source dictation utility for desktop Linux that recognizes speech locally through VOSK. It's for people who want voice input in their existing applications and are comfortable with a command-line tool. Audio processing stays on your machine, and recognition works offline.

Evidently is an Apache 2.0 Python library for evaluating, testing and monitoring ML and LLM systems. It works with tabular and text data, including predictive models and RAG applications. You can run one-off evaluations or self-host its open-source monitoring UI. Evidently Cloud is a separate hosted service.

Nanobrowser is an open-source AI agent that automates web tasks inside Chrome or Edge. It's for people who want to delegate repetitive browsing or research while choosing which models handle the work. The extension runs in your browser and is an alternative to OpenAI Operator, with an Apache 2.0 license.

Langchain-Chatchat is a self-hosted application for asking questions about your own documents and using AI agents. It focuses on Chinese-language use and open models, with a fully offline setup that can keep documents and model processing on your hardware. Its code is open source under Apache 2.0.

OpenSuperWhisper is a local speech-to-text app for people who want to dictate or transcribe recordings on an Apple Silicon Mac. It supports Whisper and Parakeet, with model downloads available inside the app. The project is open source under the MIT license.

ChatOllama is an AI agent app whose main interface is a command-line client running on your machine. It's for people who want an agent to work with local files while choosing between models served by Ollama and hosted services. The repository license modifies Apache 2.0: commercial hosting and multi-tenant services require authorization, and frontend branding must remain intact.

AstrBot brings AI assistants into messaging apps such as Telegram, Discord, Slack, QQ and WeCom. It's an open source platform under AGPL-3.0 for people building personal companions, customer support bots or team automation. You can run it on your own computer or server, including through Docker, or use its desktop app for browser-style chat.

TabbyAPI is a self-hosted LLM API server built around ExLlamaV3, for people who want local model inference behind an OpenAI-compatible API. It's the official server for that backend. The project targets personal use and small groups, and its maintainers explicitly advise against using it for production workloads.

Data-Juicer is a Python framework for preparing AI datasets on your own machine or a distributed Ray cluster. It's for researchers and teams curating model training data, agent interaction records or documents for retrieval. The project is open source under Apache 2.0.

Skills (mattpocock) is an MIT-licensed collection of engineering workflows for developers using Claude Code, Codex and other AI coding agents. Its small, separate skills work with any model and can be combined or adapted to your project, so you can choose specific practices without adopting an entire development process.

TEN Framework is a self-hosted framework for developers building voice AI agents and multimodal conversational apps. It focuses on low-latency, real-time conversations and supports both RTC and WebSocket connections. You can run its agent examples locally with Docker or deploy them on your own server.

Whisper WebUI turns audio into transcripts and subtitles through a browser interface that runs on your own machine or a self-hosted server. It's for people captioning videos, transcribing recordings or translating spoken content who want local speech processing. The project is open source under Apache 2.0 and supports Docker and Pinokio.

Cheshire Cat is a self-hosted Python framework for people learning how AI agents work or building custom assistants for research and creative projects. It pairs a local web chat interface with an API, so you can test an agent in conversation and use it inside another application. It's open source under GPL-3.0.

Mattermost Agents brings AI assistants into a self-hosted Mattermost workspace, with a choice of local models or cloud providers. It's for teams that want help with long discussions and meeting recordings while controlling where their collaboration data goes. The plugin is open source under Apache 2.0.

Distributed Llama runs a local LLM across several computers, sharing both the computation and the model's memory use. It's for people who want to use their own networked hardware for inference rather than keep the entire workload on one machine. The C++ project is open source under the MIT license.

Oterm is a terminal chat client for people who want to use LLMs without leaving their terminal. It connects to Ollama for local models and to cloud providers including OpenAI and Anthropic. Your choice of provider determines where the model runs.

Leon is a self-hosted personal AI assistant for Linux, macOS and Windows. The project focuses on the 2.0 Developer Preview on its develop branch. Its MIT-licensed runtime combines tools, context and memory with a local web interface.

RKLLM is a software stack for developers building local AI applications on Rockchip hardware. It uses the chip's neural processing unit (NPU) to run language and multimodal models on development boards, with support for the RK3588, RK3576, RK3562 and RV1126B series.

LayoutParser is an open-source Python library for developers and researchers who need to detect page structure in document images and turn OCR output into structured data. Its pretrained deep learning models share a common interface, so you can work with models trained on different document datasets without rewriting the surrounding pipeline.

localGPT is a self-hosted AI document chat app for people who want to question and summarise files on their own hardware. Its local Ollama setup keeps documents and conversations on your machine. Answers include source passages, so you can check what the model used.

OpenAdapt turns a demonstrated task into a repeatable program for browser, desktop, and remote applications. Its local, MIT-licensed open-source engine is for teams and AI agent builders who need to automate work in interfaces their APIs can't reach. It checks the outcome independently before reporting success and stops when verification fails.

Agent S is an open-source AI agent framework that controls ordinary desktop and web applications through the screen, mouse and keyboard. It runs on macOS, Windows and Linux and is aimed at developers, computer-use researchers and people building automation for their own desktops. You describe the task in natural language; the agent clicks, types and scrolls without requiring an API integration or a separate script for each application.

llm-d is an open-source stack for teams serving large language models on their own Kubernetes clusters. It coordinates model servers such as vLLM and SGLang across multiple machines, with routing and resource management for production traffic. It uses the Apache 2.0 license.

LLM is an Apache-2.0 command-line tool and Python library for sending prompts to local models and remote APIs. Local model support comes through plugins; cloud providers require their own API access. It can also connect to an arbitrary OpenAI-compatible Chat Completions endpoint, including LM Studio.

AG2 is an open-source Python framework for developers and researchers building systems where AI agents share work. The code uses the Apache 2.0 license.

Willow is a self-hosted voice assistant platform for people who want home automation voice control on their own hardware. It runs on Espressif's ESP32-S3-BOX family and connects to Home Assistant, openHAB, or other services that accept speech results over HTTP. The project is open source under Apache 2.0.

Easy Diffusion runs Stable Diffusion on your own computer through a browser interface. It's for people who want to generate and edit images locally without assembling the software components themselves. The free distribution bundles the required software and works on Windows, Linux and macOS.

Safetensors is a file format and library for developers who store, share, or load AI model weights on their own hardware or servers. It avoids the arbitrary code execution risk of PyTorch's pickle-based files while supporting fast access to tensor data. The project is open source under Apache 2.0, with a Rust implementation and Python support.

Maid is an Android AI chat app for people who want to run models on their phone and access remote models in the same app. It runs GGUF models locally through llama.cpp without an internet connection. It's open source under the MIT license, with no ads or telemetry.

CodeFormer restores degraded faces in photos and videos using a learned library of facial details and a Transformer model. It's for people restoring images on their own hardware, developers adding face enhancement to a project, and researchers working on image restoration. Its main distinction is an adjustable balance between visual quality and fidelity to the input face.

CVAT is a browser-based data annotation platform for teams building computer vision datasets. Its open-source Community edition runs on your own infrastructure with Docker and uses the MIT license. CVAT Online is hosted by CVAT, while the Enterprise offering runs in an organization's own cloud or internal environment.

TorchAO is a PyTorch library for developers who want to train or run models on their own hardware with less memory and faster computation. It reduces the precision of model weights and activations, with options for language models and image or video generation. Its PyTorch integration works with torch.compile and FSDP2 across most Hugging Face PyTorch models.

text-generation-webui, also called TextGen, runs language models on your own hardware through a desktop app or a self-hosted browser interface. It's for people who want private chat and writing tools, and developers who need a local model API. It works offline without telemetry; web search and page fetching use the internet.

Kokoro-FastAPI runs the Kokoro-82M speech model on your own machine or server and exposes an OpenAI-compatible speech API. It's for developers adding local text-to-speech to assistants, reading apps or audiobook workflows. Speech generation runs locally, and the API doesn't require an OpenAI account.

Gitingest turns a Git repository or local directory into a text digest that developers can give to an LLM as code context. It combines the directory tree and file contents in one extract, so you don't have to assemble context file by file. It's open source under the MIT license.

MuseTalk is a local AI lip-sync model for creators and developers working on video dubbing or virtual avatars. It edits the face in an existing video to match supplied speech, including Chinese, English and Japanese audio. It runs on Windows and Linux with NVIDIA GPUs, and can process videos generated by MuseV.

Triton Inference Server, offered by NVIDIA as Dynamo-Triton, is a self-hosted AI inference server for teams deploying models in applications. It serves models from different frameworks through one server, with support for on-premises hardware, cloud infrastructure and edge devices. It's open source under the BSD-3-Clause license.

PhotoPrism is a self-hosted photo and video library for people who want to organize personal media on their own hardware or server. Its AI recognizes faces and labels pictures by content and location, so finding a photo doesn't depend entirely on folders or tags you've added yourself. You browse and share the library through a web app.

macmon is a local system monitor for Apple Silicon Macs that reads hardware performance data without administrator privileges. It's for people checking resource use during local AI workloads or other demanding tasks, and developers who need those measurements in their own tools. It runs on macOS and supports M1 through M5 chips.

LangChain is an MIT-licensed open-source framework for developers building AI agents and applications powered by LLMs. It provides a shared interface for models, tools and data connections, so developers can change providers or test workflows without rebuilding the whole application.

Spring AI is an open-source Java framework for developers adding AI to Spring applications. It connects application data and APIs to models through a common interface, with Ollama support for local LLM use and integrations with cloud providers such as OpenAI, Anthropic and Amazon Bedrock. The framework runs within your application; your choice of model provider determines whether model requests stay local or go to a cloud service.