Local & Self-Hosted AI Tools

Ludwig is an open-source Python framework for developers and researchers who want to train custom AI models on their own hardware. A YAML file describes the model and training pipeline, while Ludwig handles preprocessing, training and evaluation. It uses the Apache 2.0 license. Install the Python package with the optional LLM dependencies for fine-tuning; current source requires Python 3.12 or later.

Intel Neural Compressor is a Python library for developers compressing AI models for deployment on their own hardware or servers. It supports local LLM work as well as other deep learning models, with particular attention to Intel CPUs, GPUs and Gaudi accelerators. It's open source under the Apache 2.0 license.

AutoAWQ is a Python library for developers who want to compress and run LLMs on their own hardware using 4-bit Activation-aware Weight Quantization (AWQ). The project is archived and no longer maintained. It's open source under the MIT license. It installs as a Python package, with optional kernel or Intel CPU dependencies.

H2O LLM Studio is a self-hosted tool for teams that want to adapt language models to their own datasets without writing training code. Its browser interface brings training experiments, evaluation, and model testing into one place. The project is open source under Apache 2.0.

Higgs Audio is a family of text-to-speech models from Boson AI for developers building narration and conversational audio. Higgs TTS 2 can adapt pacing and intonation to the text and generate dialogue with distinct speakers across multiple languages.

E5 Embeddings is a family of text embedding models for developers building search and retrieval systems on their own hardware. It converts text into numerical representations for matching queries with relevant passages. The family includes English and multilingual models, plus instruction-based variants for task-specific embeddings.

Open-LLM-VTuber is a local AI companion for people who want a character they can talk to, with a Live2D avatar that responds through speech and expressions. It runs on Windows, macOS and Linux through web and desktop clients. With local models for speech and language processing, it works fully offline and keeps conversations on your device. Cloud APIs are optional alternatives that send the corresponding processing to external services.

WhisperSpeech is a local text-to-speech system that uses OpenAI Whisper as the basis for generating speech. It's for developers and speech researchers who want to work with downloadable models on their own hardware. It supports voice cloning.

Zonos is an open-source text-to-speech model for people who want to generate speech and clone voices on their own hardware. It can match a speaker from a short reference recording, with controls for delivery and emotion. The code uses the Apache 2.0 license.

StyleTTS 2 is an open-source text-to-speech model for developers and speech researchers who want to generate expressive speech on their own hardware. It can choose a speaking style from the text without a reference recording, while its multispeaker model uses reference audio to reproduce a speaker's voice and delivery. The Python code uses PyTorch and carries the MIT license.

Sesame CSM is an open-source speech generation model for developers and researchers building voice applications on their own hardware. It uses text and audio inputs to generate speech, with support for conversational context and different speakers. It's a model component for applications that need spoken output.

AutoTrain trains custom machine learning models from your own data through a no-code interface. It's for people who need to fine-tune an LLM or build a classifier without writing a training pipeline. The local AutoTrain Advanced project is no longer maintained, so it won't receive bug fixes or new features.

Amica is a locally runnable interface for talking with customizable 3D AI characters. It's for people who want an animated, voiced character as the face of their AI assistant, with a choice of local LLM backends or cloud services. The project builds on Pixiv's ChatVRM.

Mikupad is a browser-based LLM frontend packaged as a single HTML file, for writers and people who want control over generated text. It connects to a model backend of your choice and supports both direct text continuation and chat with instruct models. It's open source under AGPL-3.0.

Anarlog, formerly Hyprnote, is a desktop AI meeting notetaker for people who want to keep private conversations on their own hardware. It captures audio from your device without adding a bot to the call and stays hidden during screen sharing. The app runs on macOS, Windows and Linux; its community application is open source under the MIT license.

UpTrain is an open source LLM evaluation tool for developers who need to measure answer quality and investigate failures in their AI applications. Its self-hosted web dashboard runs on your machine through Docker, with a Python package for evaluations inside application code. The dashboard requires no coding.

LLMPerf is a Python library for developers comparing LLM API performance and checking basic response correctness. The project is archived and no longer maintained. Its open-source code uses the Apache License 2.0.

Ollama Grid Search is a desktop app for comparing LLM responses across models, prompts and inference settings. It runs on macOS, Windows and Linux, and suits developers or anyone choosing a model and prompt combination for a particular task. You can inspect the responses together rather than repeat each test by hand.

Langtrace is an open-source observability tool for developers who need to debug LLM applications and track their performance. You can run it locally or on your own servers with Docker and Docker Compose. Its traces follow OpenTelemetry standards.

Pyramid Flow is an open-source AI video generator for people who want to create clips on their own hardware, and for researchers working on video models. It turns text prompts into video or animates a supplied image with guidance from text. The Python code uses PyTorch and carries MIT. Model licenses are separate: the SD3-derived weights use the Stability AI Community License. Check the chosen checkpoint before deployment.

Voxtral is Mistral AI's open-source audio and text model for developers building self-hosted speech applications. It can answer questions about recordings and produce structured summaries within the same model that transcribes speech. Mistral's separate mistral-inference library is archived and no longer maintained; Voxtral supports vLLM and Hugging Face Transformers.

Paperless-AI is a self-hosted extension for Paperless-ngx users who want automatic document sorting and chat with their archive. It requires an existing Paperless-ngx instance and runs in Docker, with a browser interface for reviewing and processing documents. The project is no longer maintained.

XTTS v2 generates speech from text using a reference voice recording or a preset speaker. It runs locally through Coqui TTS and suits developers building speech into apps, as well as researchers who want to fine-tune a speech model on their own hardware.

AnimateDiff adds text-driven animation to personalized Stable Diffusion models without requiring separate training for each model. It's for artists and developers who want to generate motion while keeping the visual style of a chosen image model. The Python implementation runs locally and includes a Gradio browser interface.

Stable Video Diffusion is an image-to-video model from Stability AI that animates a supplied still image into a short clip. It is for developers and researchers working with video generation on their own hardware. The SVD model generates 14 frames; SVD-XT is fine-tuned for 25 frames at 576×1024 resolution.

ShellGPT is an AI terminal assistant for developers and people who work with shell commands. It runs on Linux, macOS and Windows, turning plain-language requests into commands suited to your operating system and shell. You can review, explain or execute its suggestions, and its Bash and Zsh integrations put generated commands into the terminal input line for editing.

Agnai is a self-hosted AI roleplay chat app for people who want to create fictional characters and talk with them alone or in a group. A conversation can include multiple people and multiple bots. It builds on early work from Galatea-UI by PygmalionAI and uses the AGPL-3.0 open-source license.

RisuAI is an open-source AI roleplay client for people who want to create characters, build fictional worlds and chat with several characters together. It runs on Windows, macOS, Linux, Android and in a browser. You can also host the web app yourself with Docker.

tlm is an open source terminal assistant for people who want help writing shell commands and understanding unfamiliar ones. It runs models on your workstation through Ollama, so command assistance doesn't depend on a cloud service. It works offline and requires no API key or subscription.

Lunary is a self-hosted LLM observability and prompt management platform for teams building chatbots and AI agents. It brings production traces, user conversations and prompt versions into one place so developers can investigate errors and teams can assess response quality. You can host it in your own infrastructure or use its cloud service.

TensorZero is a self-hosted platform for developers building LLM applications. The project is archived and no longer maintained. It combines a model gateway with tools for inspecting responses, evaluating workflows, and improving prompts using production data and human feedback.

MindMac is a native macOS AI chat client for people who want to use local LLMs and cloud services in the same app. Its inline mode lets you ask questions or generate text inside Notes, Mail and browsers without switching to a chat window. It runs on Intel and Apple Silicon Macs with macOS 13 or newer.

TypingMind is a browser chat frontend for people who want to choose their model providers and manage conversations in one place. It connects to cloud APIs such as OpenAI, Claude and Gemini, and supports custom endpoints for locally hosted models such as Ollama and LocalAI. The frontend does not run model inference itself.

mods is a command-line AI tool for people who want to ask questions about command output or use model responses in shell pipelines. It connects to local LLMs through LocalAI as well as cloud services. The project is archived and no longer maintained.

Pieces runs in the background on your computer and builds a searchable history of your work across apps. It's for people who need to recover a decision, find earlier research or resume a task after switching between tools. Memories stay on your machine by default.

pgai keeps search embeddings in sync with PostgreSQL data for developers building RAG applications and AI agents. It's a Python library with database components and workers you can self-host, including in Docker. The project is archived and no longer maintained or supported. Its code is open source under the PostgreSQL License.

chromem-go is a vector database that runs inside your Go application, so developers can add semantic search or retrieval augmented generation (RAG) without maintaining a separate database server. It stores text alongside embeddings and retrieves related documents for use in LLM answers. Its focus is ordinary application workloads rather than collections containing millions of documents.

Quivr Core is a Python framework for developers adding document-based AI answers to their own applications. It combines file ingestion with retrieval-augmented generation (RAG), so a model can answer questions using material from your documents. It supports local models through Ollama as well as cloud APIs from OpenAI, Anthropic, and Mistral.

Cognita is a self-hosted RAG framework for developers building applications that answer questions using their own documents. The project is archived and no longer maintained. It combines a browser interface for document Q&A with reusable components built on LangChain and LlamaIndex, under the Apache 2.0 open-source license.

Backyard AI's desktop app uses llama.cpp, a runtime for running language models locally. It's for people looking for a desktop AI app on Windows or macOS, with separate Mac downloads for Intel and Apple Silicon hardware.

Open-Sora is an open source AI video generation project for developers, researchers, and creators who want to run and adapt a model on their own hardware. Its model focuses on turning reference images into video, with text prompts guiding the result. It also generates video directly from text. The code and Open-Sora 2.0 weights use Apache 2.0.

SkyReels-V2 is a family of downloadable video models for developers and researchers who want to run generation on their own GPUs. It supports text-to-video, image-to-video and extended video generation. Its autoregressive Diffusion Forcing architecture generates longer sequences by continuing video segments.

Mochi 1 is a text-to-video model for creators and developers who want to generate videos on their own hardware or adapt a model to their own footage. Genmo releases it under Apache 2.0, with downloadable weights and code for local use. Genmo also offers a hosted playground for trying the model in a browser.

5ire is a free desktop AI assistant that combines chat, a local document knowledge base and MCP tools. It runs on macOS, Windows and Linux, with Mac downloads for Apple Silicon and Intel. It's for people who want to use their own documents and external tools alongside conversations with local or cloud models.

Ollamac is a native macOS chat app for people who want to use Ollama models through a desktop interface. It works with all models supported by Ollama and can run offline through a local Ollama installation. Ollama handles the model execution; Ollamac provides the interface for conversations.

CogVideoX is a family of downloadable video generation models for developers, researchers and creators who want to generate clips on their own hardware. It turns English text prompts into video, animates a supplied image and can continue an existing video. A local Gradio web interface provides a browser front end for generation.

CogView4 is a text-to-image model you can run on your own hardware, with support for Chinese and English prompts and Chinese text within generated images. It's aimed at developers and image creators who want local AI generation with native Chinese language support. The CogView4-6B model weights and repository code use Apache 2.0.

Chroma is an 8.9B text-to-image model based on FLUX.1-schnell for creators and developers who want local image generation. The original model repository is deprecated; its maintainer recommends Chroma1-HD, Chroma1-Base or Chroma1-Flash instead. Those are separate successor checkpoints.