localGPT

localGPT is a self-hosted document chat app for people who want to ask questions about files and summarise them on their own hardware. Its local processing keeps document content and chat data on the machine running the app.

• Document questions: Ask questions in natural language and inspect the source passages returned with answers.
• File processing: Read PDF, DOCX, HTML, Markdown and text files through Docling, with OCR fallback for PDFs that lack a text layer.
• Combined search: Match both meaning and exact wording through vector search and LanceDB full-text search, then rerank the results for relevance.
• Complex questions: Split questions into sub-questions and choose between document retrieval and a direct model answer; optional sentence pruning removes irrelevant text from retrieved passages.
• Collections and conversations: Organise document collections and topic-based chats in a browser interface, with progress updates during retrieval and answer generation.

Generation uses Ollama, with model choice available per conversation or request. Embeddings can use HuggingFace models such as harrier-oss-v1 and Qwen3-Embedding, or Ollama embedding models.

The Python project is open source under MIT. Embedding and reranking can use NVIDIA CUDA, Apple MPS or the CPU. REST APIs let developers add document processing and chat to their own applications.