Local & Self-Hosted AI Tools

Clawlet is a self-hosted personal AI agent for people who want an assistant on their own machine or server. It supports local models and can retrieve past context through optional memory search.

Refact is an AI coding assistant for developers who want to delegate work within their IDE and give the agent context from their repository. The original project is archived and no longer maintained; ongoing development has moved to JegernOUTT/refact.

Evalite is a TypeScript evaluation tool for developers testing LLM-powered apps. It pairs code-based tests with a local interface for reviewing how an app responds.

LLM Guard is a Python security toolkit developers can deploy as a self-hosted API to check prompts and model responses in their applications. It provides screening and sanitization rather than running the application's LLM. The project is archived and no longer maintained, including its associated Hugging Face models.

NVIDIA RAG Blueprint is a reference framework for developers building AI question answering over enterprise data. It retrieves relevant material from your documents to ground answers and can cite the retrieved sources.

Open Deep Research is a self-hosted AI agent that searches for information and writes research reports for teams that want to choose their models and tools. The project is archived and no longer maintained.

Cognita is a self-hosted RAG framework for developers building applications that answer questions from their own documents. The project is archived and no longer maintained.

RAGs lets you create a chatbot that answers questions about a single local file or web page. It's for people who want to describe their task in plain language and review the resulting setup through a visual interface.

Intel IPEX-LLM is a library for developers running or fine-tuning language models on Intel hardware. The project is archived and no longer maintained, and Intel has identified known security issues in the code.

mods is an AI command-line tool for developers and shell users who want to ask questions about command output or pass model responses into pipelines. The project is archived and no longer maintained.

TaskWeaver is an AI agent framework for developers building assistants that analyze data through Python code. The project is archived and no longer maintained.

UI-TARS Desktop is a desktop AI agent that operates graphical apps on your computer from natural language requests. It's for people who want to delegate tasks through the interfaces they already use. The app supports fully local processing and also offers remote computer and browser operators; controlling a local computer and choosing a remote operator are separate modes.

MetaMCP is a self-hosted MCP gateway for developers and teams connecting AI clients to multiple tool servers. It lets you manage the tool collections those clients receive through shared endpoints.

Voxtral Small is an audio and text model for developers building self-hosted speech applications. The separate mistral-inference library is archived and no longer maintained; this model runs with vLLM or Hugging Face Transformers.

Chainlit's Python framework lets developers self-host conversational AI applications and provide a browser chat interface around their own application logic. Its OpenAI assistant example uses the OpenAI Python SDK’s standard connection, sending model requests to OpenAI’s cloud service. Self-hosting the chat server and supplying model inference are separate choices.

ROCm is AMD's GPU computing platform for developers running AI training, inference and scientific workloads on their own workstations or servers. It provides the software needed to use AMD GPUs for these workloads, including local LLM inference.

Pixtral is a family of Mistral models for developers building multimodal applications with downloaded weights. The associated mistral-inference library is archived and no longer maintained.

Mistral Small and Large are downloadable language models for developers building chat and text applications on their own infrastructure. Their reference library, mistral-inference, is archived and no longer maintained.