Local & Self-Hosted AI Tools

Private LLM runs AI chat entirely on your iPhone, iPad, or Mac. It's an app for people who want to use language models without sending their conversations to a cloud service. After the first model download, it works offline and requires no account. Conversations stay on-device, with no tracking or logs.

PyTorch Lightning is a Python framework for researchers and developers who want to pretrain or fine-tune models on their own hardware or in the cloud. It handles repetitive training code while leaving model logic under your control. The framework is open source under Apache 2.0.

GLaDOS is a local AI voice assistant modeled on the sarcastic character from Valve's Portal games. It's for people who want a conversational companion on their own hardware, with camera awareness and connections to home automation. The Python project is open source under the MIT license and runs on Linux and Windows. macOS support is experimental.

Nanotron is a Python library for researchers and developers who want to pretrain language models on their own datasets and GPU infrastructure. Built on PyTorch, it supports NVIDIA CUDA GPUs and training across multiple servers with Slurm. It's open source under Apache 2.0.

Hollama is an open-source LLM chat app whose interface runs entirely in your browser. It's for people who want to chat with local AI through Ollama or connect to OpenAI servers, with support for multiple server connections. The interface stores data locally in the browser; the connected server handles model requests, so where inference runs depends on the server you choose.

openWakeWord is a Python library for developers building voice interfaces that listen locally for a chosen word or phrase. It includes English models for triggers such as "hey jarvis" and "alexa", plus phrases for weather and timers. The code uses Apache 2.0. Included pretrained models use CC-BY-NC-SA-4.0, which restricts commercial use.

gptme is a self-hosted AI agent that works directly in your terminal, with access to your files and installed tools. It's for developers who want a coding assistant in their own environment, and people who want an agent for data analysis or other knowledge work. The software is free under the MIT license and doesn't require a gptme account.

Wav2Lip is a local AI lip-sync tool that changes a face's mouth movements in an existing video to match supplied speech. It's for researchers and people making academic or personal video projects who want to process their own files. Commercial use is prohibited under the project's stated terms because its models were trained on the LRS2 dataset.

docTR is an open-source Python OCR library for developers building document processing tools and researchers comparing text recognition models. It reads PDFs and images on your own hardware, locating words and recognizing their text. The library uses PyTorch and carries the Apache 2.0 license.

OpenCompass is an open-source LLM evaluation platform for researchers, model developers and teams comparing models for their applications. Its Python framework evaluates local models and cloud APIs within the same experiment, so teams can compare candidates on shared benchmarks. It uses the Apache 2.0 license.

Laminar is an open-source platform for developers who need to see why an AI agent failed and check whether a fix worked. You can self-host it with Docker or on Kubernetes, including AWS and GCP, or use its managed cloud service. It uses the Apache 2.0 license.

Nextcloud Recognize automatically categorizes media on your self-hosted Nextcloud server. It's for people who want to organize photo, video and music collections without sending sensitive data to a cloud recognition service. Image processing runs on your Nextcloud machine.

ExecuTorch is PyTorch's runtime for developers building AI into mobile apps, desktop software and embedded devices. It runs models on the user's hardware, with support for Android, iOS, Linux, macOS and Windows, as well as microcontrollers. Developers can reuse a PyTorch model across targets, though hardware-specific deployments need their own exported model files.

Model2Vec turns sentence transformers into small static embedding models that run locally on CPU. It's for developers who need text embeddings for retrieval, code search or classification without the size and inference cost of the original transformer. The Python package is open source under the MIT license.

Kong Gateway puts API, LLM and MCP traffic behind a shared gateway on your own infrastructure. It's for platform teams that need consistent access controls and traffic policies across services and AI applications. The open-source gateway uses the Apache 2.0 license and runs natively on Kubernetes through Kong's official Ingress Controller.

Whishper is a self-hosted speech-to-text app for people who need transcripts or translated subtitles from audio and video. Its browser interface brings transcription, translation and subtitle editing together, with all three running on your own machine. It can work offline, so local media doesn't need to go to a cloud transcription service.

Musubi Tuner is a Python toolkit for training LoRA adapters for image and video generation models on your own hardware. It's aimed at people who want to customize these models using their own datasets and are comfortable working with training scripts. It also includes image and video generation scripts for supported architectures.

EchoMimic turns a portrait image and an audio recording into an animated talking-head video on your own hardware. Its distinguishing feature is editable facial landmark control: you can drive animation with audio, landmarks, or audio combined with selected landmarks. It's intended for academic research and suits people comparing methods for speech animation and facial motion control.

Page Assist brings local AI chat into your browser, with a sidebar for conversations alongside a webpage and a separate tab for a ChatGPT-style interface. It's for people who already run models on their own hardware and want to ask questions about what they're reading without leaving the page.

Colossal-AI is a Python framework for developers and researchers training or serving large AI models on their own GPU hardware. It addresses the memory and computing demands of models that are difficult to fit on a single GPU, with tools for distributing work across a cluster. It's open source under Apache 2.0.

HuggingChat UI is the open-source chat application behind Hugging Face's hosted HuggingChat. You can run it on your own computer or server and connect it to a local LLM backend or a cloud provider. It's for people and teams who want a browser-based ChatGPT alternative with control over the chat service and where its data lives. The code uses the Apache 2.0 license.

Blinko is a self-hosted AI note app for quickly capturing thoughts and retrieving them later. Notes and data are stored in your own environment, with plain-text storage and Markdown formatting. The project is licensed under GPL-3.0.

Meme Search is a free, self-hosted web app for people who want to find memes in their own collection by image content or text. It runs locally through Docker and uses AI descriptions to make images searchable, even when their filenames aren't useful. The code uses the Apache 2.0 license.

diffusion-pipe is a local diffusion model training tool for people fine-tuning image and video models on their own GPU hardware. Its main distinction is that it can divide a model across several GPUs when it won't fit on one, while also distributing training work across GPUs. The Python project is open source under GPL-3.0 and uses DeepSpeed.

F5-TTS is a local text-to-speech system that uses a reference recording to generate new speech in that voice without training a separate model for each speaker. It's for developers, speech researchers, and creators who want to generate voices on their own hardware. Its Python code uses MIT, while pretrained models use the noncommercial CC-BY-NC license.

TagGUI is a desktop app for people preparing image datasets for generative AI training. It combines local AI captioning with manual tag editing, so you can generate descriptions and correct them in the same workspace. It's open source under GPL-3.0 and runs on Windows, Linux and macOS, though macOS doesn't have a packaged release.

SkyPilot is an open-source system for AI teams that need to run training, inference and development workloads across their own clusters and cloud accounts. It brings Kubernetes, Slurm and cloud compute under one interface, so teams can move jobs between providers without rewriting their workload code.

xberg, formerly Kreuzberg, is a local document extraction engine for developers building AI search, document processing, and retrieval-augmented generation applications. It reads PDFs, Office files, scanned images, email, and nested archives, extracting text, tables, images, and metadata through one shared engine. It's open source under MIT.

CopilotKit is a self-hostable SDK for developers building AI agents into web and mobile apps, Slack, or Microsoft Teams. Agents can display interactive charts and forms using an app's own components, read shared app state, and take actions through frontend tools, APIs, or MCP tools.

GPTQModel is a Python toolkit for developers compressing LLMs and running them on their own hardware or servers. It brings model calibration, compression, quality checks and inference into one API, so teams can compare quantization methods without adopting a separate tool for each one.

MusicGen is Meta AI's music generation model within AudioCraft, a PyTorch library for developers and audio researchers who want to generate music in their own computing environment. It creates music from text descriptions and can use a melody to guide the result. AudioCraft includes both inference and training code, so it's suited to people building audio tools or studying music generation.

Unmute adds spoken conversation to text LLMs using Kyutai's speech recognition and speech synthesis models. It's for developers who want a self-hosted voice interface while keeping their choice of language model. The project uses the MIT license, and a hosted browser demo is available at Unmute.sh.

Moonshine is an on-device AI toolkit for developers building voice agents and applications that listen and speak. It combines speech to text, intent recognition and text to speech in one library. Voice processing stays on the device, and you don't need an account or API keys.

Fullmoon is an open source chat app for people who want to run language models on their Apple devices and keep conversations local. It supports iOS, iPadOS, macOS and visionOS, with on-device inference optimized for Apple silicon. You can chat fully offline, and the app saves your chat history locally.

vLLM Production Stack is an open source inference stack for teams serving LLMs on their own Kubernetes GPU clusters. It brings request routing and monitoring around vLLM, so applications can move from one serving instance to a distributed deployment without changing their code. It requires a GPU-enabled Kubernetes environment.

Giskard is an open-source Python library for testing AI agents, paired with a commercial security platform and assessment service. It's for developers checking agent behavior and security teams assessing deployment risks. The library runs in your own environment under Apache 2.0; Giskard Hub is available hosted or on-premise.

sd-scripts is a collection of Python scripts for training and generating images with models on your own hardware. It's aimed at people who want to customize image models through LoRA training or deeper fine-tuning and are comfortable working with scripts. The project is open source under the Apache 2.0 license.

IntelliBar is an AI assistant for Mac that brings local models and cloud services into one desktop app. It's for people who want to write, proofread, or ask questions without switching between separate chat apps. Ollama support lets you run models locally and keep prompts away from remote providers.

SenseVoice is a local speech recognition model that adds language, emotion and sound-event tags to transcriptions. It's for developers building voice applications or analyzing recordings on their own hardware, particularly those working with Mandarin and Cantonese. The project is open source under the MIT license.

MTEB is an Apache 2.0 Python toolkit for evaluating embedding models and retrieval systems. It runs evaluations through Python or a command-line interface and publishes an interactive leaderboard.

Mochi Diffusion is a native macOS image generator for people who want to create images on an Apple Silicon Mac without sending prompts or results to a cloud service. It runs Stable Diffusion and FLUX.2 Klein locally, with completely offline generation. The app collects no telemetry or other data and is open source under GPL-3.0.

MeloTTS is a Python text-to-speech library for developers who want to generate speech locally, including on machines without a dedicated GPU. It supports real-time inference on a CPU. Its language and accent choices make it relevant for applications that need spoken output across different audiences.

OpenLIT is a self-hosted platform for developers who need to understand how their LLM applications and AI agents behave. It connects model calls with tool activity, retrieval and agent steps, so teams can investigate errors and compare cost, latency and output quality across a workflow.

Paperless-GPT adds AI text extraction and document organization to an existing paperless-ngx library. It runs in Docker on your own server and suits people who want less manual sorting of scanned paperwork. The project is open source under the MIT license.

MacWhisper is a native macOS transcription app for people working with interviews, lectures, meetings and other recorded audio. It runs speech recognition on your own Mac, so local transcription keeps audio on your device. It also offers cloud transcription through services such as OpenAI, ElevenLabs and Deepgram, which send audio off your machine.

Higress is a self-hosted AI gateway for developers and teams managing model APIs and the tools their AI agents call. It puts LLM traffic and MCP servers behind a shared entry point, with authentication, traffic controls and monitoring. The open-source edition uses the Apache 2.0 license and runs locally in Docker without registration. Alibaba Cloud also offers a fully managed gateway.

Infinity is a self-hosted database for developers building search and retrieval-augmented generation (RAG) into LLM applications. It combines embedding search with full-text search and structured filters, so an application can retrieve relevant records through both meaning and exact terms.

AIBrix is open-source infrastructure for teams serving large language models on their own Kubernetes clusters. It focuses on the work around inference: directing requests, scaling capacity and managing models across servers. Enterprise infrastructure teams can use its components to build a self-hosted model service. It's licensed under Apache 2.0.