RAG Frameworks, Parsers and Vector Databases

Infinity is a self-hosted database for developers building search and retrieval-augmented generation (RAG) into LLM applications. It combines embedding search with full-text search and structured filters, so an application can retrieve relevant records through both meaning and exact terms.

Zerox is an MIT-licensed OCR library for developers preparing documents for AI applications. Its Node.js and Python packages run on your own machine or server, while cloud vision models read the document pages and produce Markdown. Document conversion happens locally, but page images go to the selected model provider, so this workflow needs internet access and provider credentials.

SurrealDB is a self-hosted database for developers building AI agents, knowledge graphs and applications that need several kinds of data together. It stores documents, relationships, vectors and time-series data in one engine, so an application's records and its AI retrieval layer can share the same database.

DeepSeek-OCR is an open-source OCR model for developers building document processing tools and researchers studying how AI reads text through images. It runs on your own hardware with NVIDIA CUDA GPUs. Its distinctive focus is visual text compression: representing document images with compact sets of vision tokens for a language model to read.

rerankers is a Python library for developers building search and retrieval systems who want to compare reranking models without rewriting their integration each time. It takes a query and candidate documents, then ranks their relevance through a shared interface across local models and hosted services. It's open source under Apache 2.0.

PDF-Extract-Kit is a local AI model toolbox for developers and researchers building document processing applications. It extracts text, tables and mathematical formulas from PDFs, with separate models for identifying page elements and recognizing their contents. It's open source under AGPL-3.0, written in Python, and supports CPU or GPU execution on your own hardware.

MonkeyOCR is a local AI document parser for developers and researchers working with English and Chinese PDFs or images. It extracts text, formulas and tables while identifying page structure and relationships between blocks. That makes it useful for documents where plain text extraction loses reading order or separates content from its layout.

LLPhant is an MIT-licensed PHP framework for adding language models, embeddings and vector databases to Symfony and Laravel applications. It requires PHP 8.1 or later and is installed through Composer.

Langchain-Chatchat is a self-hosted application for asking questions about your own documents and using AI agents. It focuses on Chinese-language use and open models, with a fully offline setup that can keep documents and model processing on your hardware. Its code is open source under Apache 2.0.

TabbyAPI is a self-hosted LLM API server built around ExLlamaV3, for people who want local model inference behind an OpenAI-compatible API. It's the official server for that backend. The project targets personal use and small groups, and its maintainers explicitly advise against using it for production workloads.

Data-Juicer is a Python framework for preparing AI datasets on your own machine or a distributed Ray cluster. It's for researchers and teams curating model training data, agent interaction records or documents for retrieval. The project is open source under Apache 2.0.

LayoutParser is an open-source Python library for developers and researchers who need to detect page structure in document images and turn OCR output into structured data. Its pretrained deep learning models share a common interface, so you can work with models trained on different document datasets without rewriting the surrounding pipeline.

localGPT is a self-hosted AI document chat app for people who want to question and summarise files on their own hardware. Its local Ollama setup keeps documents and conversations on your machine. Answers include source passages, so you can check what the model used.

Gitingest turns a Git repository or local directory into a text digest that developers can give to an LLM as code context. It combines the directory tree and file contents in one extract, so you don't have to assemble context file by file. It's open source under the MIT license.

LangChain is an MIT-licensed open-source framework for developers building AI agents and applications powered by LLMs. It provides a shared interface for models, tools and data connections, so developers can change providers or test workflows without rebuilding the whole application.

Spring AI is an open-source Java framework for developers adding AI to Spring applications. It connects application data and APIs to models through a common interface, with Ollama support for local LLM use and integrations with cloud providers such as OpenAI, Anthropic and Amazon Bedrock. The framework runs within your application; your choice of model provider determines whether model requests stay local or go to a cloud service.

FAISS is an open-source library for developers who need to search and cluster vector data on their own machines or servers. Its main strength is the choice between exact results and approximate searches that use less memory or return results faster. It's MIT-licensed.

Crawl4AI is a self-hosted web crawler and scraper for developers building AI agents, retrieval-augmented generation (RAG) systems and data pipelines. It turns web pages into Markdown or structured JSON and runs as a Python library or a Docker server on your own hardware. The open-source code uses the Apache 2.0 license.

Chroma DB is an open-source search database for developers building AI apps and agents that need to retrieve information from their own data. You can run it locally or host it on your own infrastructure under the Apache 2.0 license. Chroma Cloud is a separate hosted service for managed, serverless search.

kotaemon is a self-hosted document chat app for people who want to ask questions across their files and check where the answers came from. It runs in a browser on Windows, macOS or Linux, with Docker also supported. The project uses the Apache 2.0 license.

Typesense combines typo-tolerant site search with vector and semantic search in a self-hosted engine. It's for developers building searchable apps, product catalogs or AI search over their own data. The C++ engine uses an in-memory architecture for low-latency results as users type.

LanceDB is an open source vector database for developers building AI retrieval applications and teams working with training datasets. Its embedded library runs locally or in your own cloud under the Apache 2.0 license. Cloud and Enterprise offerings provide managed infrastructure for production workloads.

DocsGPT is an MIT-licensed, open-source platform for teams that want AI search, assistants and agents over their own documents. It can run on your servers with local models, including fully air-gapped deployments where documents and questions stay inside your network. Answers include the source title and page number so readers can check the evidence.

pgvectorscale adds an index for large embedding datasets to PostgreSQL databases that use pgvector. It's for application developers and database administrators who want to keep AI similarity search in their existing database, with more control over search speed and storage use.

Text Embeddings Inference is a self-hosted server for developers who need text embeddings for search and retrieval applications. It serves models through a REST API on your own hardware and can run offline once model weights are downloaded. The Rust project is open source under Apache 2.0.

PrivateGPT is a self-hosted API layer for developers building AI applications around local models. It adds document retrieval, database access and agent tools to an existing model server. Local workflows can work offline and keep data within your environment; web search and connections to online providers need internet access.

txtai is a Python framework for developers building search applications, chat with their data, and AI agents on their own hardware or servers. Its embeddings database combines sparse and dense vector search with graphs and relational data, so the same system can find related content and supply context to language models. It's open source under Apache 2.0.

DSPy is a Python framework for developers building AI applications whose tasks need clear inputs, predictable output types, and measurable results. You define what a language model should produce, then compose those tasks into a larger program. It's open source under the MIT license.

GraphRAG builds a knowledge graph from text so an LLM can answer questions that depend on connections across documents or themes across a whole collection. It's for developers and researchers working with private datasets, such as business documents, proprietary research, or communications.

Tiledesk is a self-hosted platform for building AI agents and connecting them to human support teams. It's aimed at businesses automating customer conversations, internal information searches and workflows through a visual builder. You can run it on your own server with Docker or Kubernetes, or use its hosted cloud service.

Jina Reader turns web pages and documents into text that LLMs can use, with Markdown or JSON output. It's for developers building AI agents, search tools and systems that answer questions using retrieved documents. You can self-host the Apache 2.0 service code in Docker or use Jina's hosted API.

OpenSearch combines document and enterprise search with vector retrieval for AI applications. It's for developers building search into their products and teams analyzing application logs, infrastructure performance, or security events. The suite uses the Apache 2.0 license throughout, including its data ingestion and dashboard components.

Xinference serves language, speech and multimodal models through a shared API on your own computer or servers. It's an open source platform under Apache 2.0 for developers and researchers who want to build applications around models they host. You can also deploy it on cloud infrastructure.

Meilisearch combines keyword search and AI retrieval in a search engine you can host on your own server. It's for developers building search into websites, applications, product catalogs, or internal data tools. Meilisearch Cloud provides a separate, fully managed hosted service.

Marker is a local document converter for developers and teams turning PDFs, scans and Office files into structured text. It preserves tables, equations and page structure for document processing and AI workflows. Its pipeline reads embedded PDF text and uses Surya OCR where text is missing or damaged, rather than reading every page through a vision model.

sqlite-vec adds vector storage and similarity search to SQLite, so developers can keep embeddings alongside application data in a local database. It's for applications that need to find related items by vector distance without running a separate vector database server. The extension is small, written in C and has no dependencies.