Local & Self-Hosted AI Tools

LiveKit Agents is a framework for developers building voice assistants, phone agents and applications that combine speech, video and text. Python and Node.js agents interact with users through LiveKit rooms connected to web, mobile or telephone clients.

llama.cpp is a local LLM inference engine for developers building AI applications or running models on hardware they control. It supports language and vision-language models, with command-line tools and a server API for using the engine directly or connecting other software.

Triton Inference Server, offered by NVIDIA as Dynamo-Triton, is a self-hosted AI inference server for teams deploying models in applications. It brings models from different frameworks into one serving system across on-premises, cloud and edge hardware.

DeerFlow is a self-hosted AI agent application for research, coding and content creation on your own infrastructure. It supports local models through Ollama and vLLM, including Qwen3, as well as hosted providers. With a cloud provider, model requests leave your server and may incur provider charges.

Flower is a federated AI framework for developers and researchers who want to train models across separate organizations or devices. Participants retain control of their data rather than gathering everyone's data in one place.

InsightFace is a face and person analysis toolkit for developers building identity verification, access control and media applications. Its Python tools and recognition server can process data on your own hardware.

Soup is a local LLM fine-tuning tool for developers and teams adapting models to their own datasets. It brings training, evaluation and export into one workflow that runs on your own hardware without requiring a cloud service.

Manifest LLM Gateway connects AI agents and apps to local models, cloud providers and subscriptions through a shared endpoint. It's for builders and teams who want to choose where requests go, handle provider failures and track inference usage.

deja-vu helps developers and coding assistants recover fixes, decisions and unfinished work from session history already on disk. It can search conversations recorded before installation and carry relevant context between Claude Code, Codex CLI, Cursor and opencode.

Coze Studio is a self-hosted AI agent development platform for teams building assistants and apps through visual, low-code tools. It brings creation, debugging and publishing into one workspace.

llama-cpp-python is for developers adding local LLM inference to Python applications or connecting existing API clients to models on their own hardware. It brings llama.cpp into a Python library and a self-hosted server.

Pictaria Server helps Immich users explore their photo library, choose photos to keep and organize albums through a browser dashboard on their own hardware.

Radiant Camera is an iPhone camera and photo editor for people who want automatic scene adjustments with room to fine-tune the result. All image processing, including AI, runs on your device. It combines capture and editing in one app, with scene detection for portraits, pets, macro shots, night scenes, and landscapes.

AI Legion is a self-hosted framework for developers who want several AI agents to work together on tasks. The program runs on your own machine through Node.js, while OpenAI's cloud models handle the language processing. It's open source under MIT.

MindWalk helps developers review coding-agent sessions by showing where an agent explored and changed a repository. It turns Claude Code, Codex and pi logs into a visual replay for comparing the agent's work with the task you assigned.

MS Agent is an open source Python agent framework with a self-hosted web server for working on local projects. It's for developers building research assistants and coding workflows that continue across sessions. Code execution and project file work run locally; the default model integration uses ModelScope's API and requires an API key and internet access.

Qube is a desktop AI assistant for people who want to chat with models and work with private documents on their own hardware. It combines a document library with persistent memory and research tools that provide citations you can inspect.

GROBID turns scientific and technical PDFs into structured TEI XML on your own machine or server. It's for researchers and developers who need article metadata, full text and bibliographical references for search, analysis or document collections. The software is open source under Apache 2.0.

RocketRide is an AI pipeline builder and runtime for developers creating document processing, search, and agent workflows. Its visual editor sits in VS Code, alongside the application code that uses those workflows.

NVIDIA NeMo Agent Toolkit is an open-source Python library for developers building self-hosted servers that expose AI tools and workflows through MCP or A2A. It also measures and evaluates agent workflows across frameworks. Its introductory NVIDIA NIM workflow uses hosted inference and requires an NVIDIA account and API key.

OpenDev is an AI coding agent for developers who want concurrent work across a codebase and control over which models handle each task. It can plan changes, execute them and iterate autonomously, with a terminal interface and a browser interface for monitoring work.

LangChain4j is a Java library for building chatbots, assistants and AI agents that use local models through Ollama. It runs within JVM applications, while Ollama handles model inference on your hardware using a CPU or GPU. You can also connect to OpenAI or Google Vertex AI, in which case model requests go to the remote provider.

SD.Next is a self-hosted web interface for artists and researchers who want to create and edit AI images and videos on their own hardware. It supports Stable Diffusion and other diffusion models.

Argilla is a data annotation and feedback tool for AI engineers and domain experts building training and evaluation datasets. It gives reviewers a shared interface for checking examples and contributing judgments that inform model development.

Axolotl is a free, open-source LLM fine-tuning framework for developers and researchers adapting models to their own data. It supports training on local hardware or cloud infrastructure you control.

Sandbook is an offline text-to-speech reader for iPhone and iPad, for people who want to listen to books and documents without uploading them. It runs Kokoro-82M on the device through Apple MLX, with the speech model and voices included in the app. It needs no account and has no ads or tracking.

AI Documentation Generator analyzes repositories and writes documentation for developers who need to understand an unfamiliar codebase or prepare it for new contributors. It's open source under the MIT license and runs as a Python application, in Docker, or through a Claude Code plugin.

CutScript is a desktop audio and video editor for podcasters and creators who prefer editing speech through text. Deleting a word from the transcript cuts the matching section of the recording, making it a local-first Descript alternative.

TagGUI is a desktop app for creators preparing image datasets for generative AI models. It brings automatic captioning and manual tag editing into one workspace so you can review and correct generated descriptions.

Mobile MCP connects AI agents to native iOS and Android apps for testing, form entry and extracting data from screens. Developers and testers can use the same tools across physical devices, simulators and emulators.

Munder Difflin is a desktop app for developers who want coding agents to collaborate on their own machine. You give an orchestrator a request, and it divides the work among agents instead of leaving you to coordinate separate terminals.

Whisper Notes transcribes interviews, lectures and meetings for people who need recordings to stay on their own device. It works without an internet connection or account.

OpenSpec helps developers agree on what an AI coding assistant should build before it edits code. It keeps requirements and implementation plans in project files so they don't depend on chat history, and supports both existing codebases and new projects.

Graphiti is a Python framework for developers building AI agents that need to remember information as it changes. It turns conversations, structured records and unstructured text into a queryable graph with a history of facts.

AI Review is an AI code review tool for development teams that want automated feedback inside pull and merge requests. It runs in your CI/CD environment and uses your chosen LLM to review code against your team's guidelines.

E5 Embeddings is a family of text embedding models for developers building search and retrieval systems. It represents text as numerical vectors so applications can match queries with relevant passages on their own hardware.

PocketFlow is a Python framework for developers building AI agents and workflows with their own model connections. Its utility examples include an Ollama function for local LLM calls. Developers supply that function rather than using a built-in model wrapper. Cloud API examples send model requests to their providers. FastAPI examples show chat applications with streamed responses and background jobs.

WhisperLiveKit is a self-hosted speech-to-text server for meeting transcripts, live captions and apps that need speech recognition as people talk. It processes audio on your own machine or server.

Askimo is an AI desktop app for people who want to ask questions about their files and delegate work from the same place they chat. It supports local LLM backends and cloud providers, including Ollama, LM Studio, OpenAI, Claude and Gemini.

Depth Anything estimates depth from ordinary images for developers and researchers who need depth maps without a depth camera. Its pretrained models show which parts of a scene are nearer or farther away; separate fine-tuned models estimate metric depth.

PaddleSeg is an image segmentation toolkit for developers building computer vision applications on their own hardware. It covers annotation, training and deployment for tasks such as medical image analysis, industrial inspection and portrait effects.

Toonflow is an AI creation platform for short dramas, animated stories and short videos. It takes novels or brief ideas through scriptwriting, storyboards and video production in a shared canvas workspace.

smolagents is a Python AI agent library for developers building software that uses tools and calculations to complete tasks. Its CodeAgent writes and executes Python actions, combining tool calls with loops and conditional logic.

MindMac is a native macOS AI chat client for people who want local models and cloud providers in one app. Its inline mode lets you get writing help inside Notes, Mail and browsers without switching to a separate chat window.

fcitx5-vinput adds voice typing to Fcitx5 on Linux. It's for people who want to dictate text and use spoken instructions to edit it, with a choice between local speech recognition and cloud services.

PaddleOCR is an OCR and document parsing toolkit for developers turning PDFs and images into data for document search, RAG systems and AI agents. Its local models cover both structured pages and text found in photographs.