Local & Self-Hosted AI Tools

FlashInfer is a GPU kernel library for developers building LLM inference systems on their own hardware or servers. It supplies the attention, matrix multiplication and mixture-of-experts operations used by model serving software.

vLLM Production Stack is an open source LLM serving stack for teams running models on Kubernetes GPU clusters. It lets applications move from a single vLLM instance to a distributed deployment without changing application code.

Lingolay is a free, open source screen translator for people who want translated subtitles while playing games or watching videos. It reads visible text with OCR, translates it on your computer and displays the result in a floating overlay. Optional Piper voices read the translations aloud.

MLX Tune is a Python library for adapting AI models on an Apple Silicon Mac. Formerly unsloth-mlx, it is an independent project and is not affiliated with Unsloth. Its Unsloth-compatible API lets you prototype training locally with minimal changes to existing scripts, then move to NVIDIA hardware with Unsloth. Training runs through Apple's MLX framework and doesn't require a cloud GPU.

MaxText is an LLM training library for researchers and engineers building models from scratch or adapting existing ones. It provides a reference implementation teams can modify for their own model designs, with training that scales from a single host to large clusters.

Unmute gives text LLMs a spoken conversation interface using Kyutai speech recognition and speech synthesis. It's for developers who want to self-host voice chat while choosing their own language model.

DINOv3 is Meta AI's family of vision foundation models for researchers and developers building image analysis systems on their own hardware. Its pretrained models extract features for individual image regions that can support multiple vision tasks without fine-tuning the backbone.

TrustLLM is a research toolkit for teams comparing language models beyond answer quality. It evaluates local LLMs and models accessed through APIs for behaviors such as hallucination, harmful responses, bias, and information leakage.

OpenBrowserAgent is an installed browser extension for Chrome, Firefox and Safari that combines page-focused chat with browser automation. It connects to local Ollama endpoints or model APIs you configure. Requests send selected browser context and messages to that endpoint; a remote model connection sends that material outside the browser.

DocNavigator is a self-hosted AI chatbot builder for teams that want customers to ask questions about their product documentation or support content. It uses information from a company's website as the basis for the chatbot, so responses can draw on that company's material.

Jina Embeddings is a family of models for developers building semantic search, retrieval-augmented generation (RAG) and AI agent applications. The models convert content into vectors that a search system can compare for relevance.

TypingMind is an AI chat frontend for people who want to use different model providers in one workspace. It brings conversations, project materials and custom AI agents into a shared interface.

Oriveo Community Edition is a free, open-source AI chat client for people who want local models and cloud providers in the same app, with their conversation history stored on their own device. It runs on iOS, Android and the web, and you can self-host the web client. It doesn't require an Oriveo account.

VoiceStreamAI turns live browser audio into text on a server you control. It's for developers adding streaming transcription to an app or building a speech-to-text service around Whisper.

roborev reviews commits in the background for developers working with AI coding agents. It brings findings back into active coding sessions so bugs can be addressed before a pull request.

LibreDB Studio is a self-hosted database IDE for developers and database teams who want browser access to data inside their own network. It can run beside the database, with an interface that also works on phones and tablets.

Codex Router runs on your computer and connects the Codex App and CLI to external model providers or your own self-hosted endpoints. It's for developers who want a different model within their existing coding workflow. Cloud routes send model requests to remote services and need internet access; paid providers still charge for access or usage.

ReMe gives AI agents lasting memory across conversations. It's for developers who want coding assistants or personal agents to retain useful facts, preferences, and procedures in files they can inspect and edit.

AudioToText turns recordings into text and captions for people who need transcripts or subtitles in different languages. It can process audio on your computer with Whisper or use cloud processing.

Alfred Black is a self-hosted AI agent for people who need help tracking messages, tasks and ongoing commitments. It runs continuously and brings related information together so you can review a concern without piecing together scattered threads.

WhisperWriter turns microphone speech into typed text in your active window. It's for people who want dictation in their desktop apps, with a choice of transcription on their own computer or through an API.

CVAT is a data annotation platform for teams preparing datasets for computer vision and audio models. It combines labeling tools with shared assignments and review workflows.

picoLLM is an on-device LLM inference SDK for developers adding text generation to their apps. Inference works offline, but a Picovoice account and AccessKey are required, and key validation needs an internet connection.

Claude Code Router is a local gateway for developers using coding agents such as Claude Code, Codex and OpenCode. It keeps provider selection and request routing in one app, while the selected cloud provider processes model requests remotely. Those requests leave your machine.

Vidurai is a local AI memory service for developers who want their AI assistant to understand recent work without repeatedly explaining it. It tracks file changes, terminal activity and errors, then prepares a focused summary for an assistant. The background service and its stored development history stay on your machine.

LangChain is an open-source framework for developers building AI agents and LLM applications, including apps that run models locally through Ollama. Its Python ChatOllama integration connects to open-weight models such as gpt-oss. You write the application in code, while Ollama handles local inference.

SillyTavern is a locally installed interface for AI hobbyists who want control over prompts and access to text, image and voice models in one place. It's an independently developed fork of TavernAI and connects to a separate model backend or API provider.

Coze Loop helps developers test and monitor AI agents, with tools for investigating failed runs and comparing prompt changes. Its self-hosted edition brings prompt development, evaluation, and execution traces into one browser interface.

Examor is a self-hosted study app for students, interview candidates and people who want regular practice from their own notes. The app runs on your machine through Docker and opens in a browser. GPT generates tests and assesses answers; local hosting describes the study application, rather than a local model runner.

LM Format Enforcer is an open-source Python library for developers who need a local LLM to return structured data their applications can parse. It constrains text during generation, so a prompt's formatting request doesn't have to carry the whole burden. The library uses the MIT license.

Depth Pro estimates distances from a single image for researchers and developers who need depth maps with an absolute scale. It can make predictions without camera metadata, so images don't need to come with known camera intrinsics.

Whisper WebUI turns recordings into transcripts and subtitles through a browser interface running on your own machine. It's for people captioning videos or translating speech who want local audio processing.

PicApport is a self-hosted photo server for families and businesses that want to organize and search a shared media archive on their own hardware. Its browser gallery handles photos, videos and PDFs, with plugins for camera RAW files and other formats.

Hippo Memory gives coding agents persistent memory across projects, sessions and tool changes. It's for developers who want agents to retain useful lessons and stop recalling facts that have proved wrong.

Foreman is a terminal app for developers who want to supervise Claude Code agents working in a repository. It takes feature requests through planning, design review and implementation, with human approval before agents begin the build.

Logseq is a local-first knowledge base for people who want to organize notes and research alongside their tasks. It supports Markdown and Org-mode for working with written material.

MusicGen is Meta AI's model for generating music from written descriptions. It's part of AudioCraft, a PyTorch library for developers and audio researchers building music generation tools in their own computing environment.

GLM-4.5 is an open-source language model family for developers building AI agents and coding tools on their own servers. It supports reasoning and tool use, but its supplied inference configurations require server-class NVIDIA hardware.

MCP Context Forge is a gateway and registry for teams connecting AI clients to multiple tools, agents and services. It gives them one place to discover available services, control access and inspect calls.

OpenCompass is an LLM evaluation framework for researchers and developers comparing models for their applications. It lets you assess models on your own hardware alongside cloud APIs in the same experiment.

Trilium Notes is a note-taking app for people building large personal knowledge bases. Its tree structure lets one note appear in multiple branches, so you can organize related material without keeping separate copies.

AnythingLLM is an AI assistant for people who want document chat and agents on their own hardware. Its desktop app can work with local models without an account or cloud API key.

chromem-go is a vector database for Go developers adding semantic search or retrieval augmented generation (RAG) to an application. It runs inside the app, so it doesn't need a separate database server.

Noi is an AI browser for people who work across AI chats and want to keep separate tasks organized. It combines multiple workspaces with reusable prompts and access to local tools.

Vearch is a self-hosted, distributed vector database for developers building AI applications that search embedding vectors. It combines similarity search with filters on stored fields, so an application can retrieve related content while applying criteria beyond vector similarity. It's open source under Apache 2.0 and can run on your servers through Docker, Docker Compose, or Kubernetes.

Izwi is a local voice AI app and runtime for people and teams processing speech on their own hardware or inside infrastructure they control. It brings audio tools and voice conversations into a desktop app, browser interface, CLI and inference server.