localGPT

localGPT is a self-hosted AI document chat app for people who want to question and summarise files on their own hardware. Its local Ollama setup keeps documents and conversations on your machine. Answers include source passages, so you can check what the model used.

It handles PDF, DOCX, HTML, Markdown and plain text through Docling, with OCR for PDFs that lack a text layer. The browser interface lets you organise document collections and keep conversations by topic. It also shows progress while the app searches your files and generates an answer.

Search combines meaning-based retrieval with LanceDB full-text search, then reranks the matches for relevance. This lets it look for both related ideas and exact wording. It can break complex questions into smaller questions and choose between consulting your documents or answering directly with the model. Optional sentence pruning removes irrelevant material from retrieved passages.

Ollama provides the generation models, with Qwen models used by default. You can choose different models for individual conversations or messages. Embeddings can come from HuggingFace models such as harrier-oss-v1 and Qwen3-Embedding, or from Ollama.

The MIT-licensed project runs on macOS, Windows and Linux, with Docker deployment available. It requires at least 8GB of RAM and recommends 16GB or more. Models need an initial download; embedding and reranking can use NVIDIA CUDA, Apple MPS or the CPU. Developers can use its REST APIs to add document indexing and chat to their own applications.