AnythingLLM

AnythingLLM is an open source AI assistant for people who want to chat with their documents and use AI agents on their own hardware. The desktop app runs on Windows, macOS and Linux, while a Docker deployment supports multiple users on a self-hosted server. No account is required for the desktop app.

You can bring PDFs, text files and Word documents into a workspace, then ask questions with source citations. Agents can use document knowledge, browse the web and carry out scheduled tasks. AnythingLLM also has memory, custom agent skills and a builder for creating agents without code. Its developer API and embeddable chat widget give teams ways to use those workspaces in other applications; the widget is part of the Docker version.

Model choice is flexible. AnythingLLM works with local backends such as Ollama, LM Studio, LocalAI, llama.cpp-compatible models and Microsoft Foundry Local, alongside cloud providers including OpenAI and Anthropic. It can select models dynamically for different conversations. On Windows, Foundry Local can use a CPU, GPU or NPU. Cloud models and web search use external services, while a fully local model and embedding setup can keep document processing on the device.

The desktop meeting assistant records, transcribes and summarizes calls locally, without a meeting bot joining. Dictation and text tools also run on device. An Android app runs small models on the phone and can connect to a desktop installation for access to larger models and its workspaces. The project uses the MIT license. Its optional telemetry reports usage events such as chats and document changes, without sending their contents.