RAG Frameworks, Parsers and Vector Databases

DB-GPT is a self-hosted AI data assistant for teams analyzing business data and developers building data applications. It connects plain-language questions to database queries and code-based analysis.

rerankers is a Python library for developers building search and retrieval systems. It ranks candidate documents against a query and lets you compare reranking approaches through a shared interface.

AutoRAG Agent searches documents and connected archives where they already live, then reads the relevant sources to produce answers with file, page or line references. It runs on macOS, Windows and Linux and suits people researching information spread across local files, messages and notes.

Data-Juicer is a Python framework for researchers and teams preparing datasets for AI training, evaluation, agents or document retrieval. It processes data on your own machine or across a Ray cluster.

FAISS is a vector search and clustering library for developers building applications that compare embeddings or other vector data. It runs on your own machine or server and lets you balance search accuracy against speed and memory use.

KaaS is a self-hosted AI knowledge base for personal research and teams preserving knowledge across role changes. It turns notes, documents and transcripts into wiki articles you can read, edit and query.

RAGFlow helps teams build AI agents that answer questions using their own documents. It extracts content from complex files and retrieves supporting material that readers can check.

Chandra OCR converts document images and PDFs into text that retains page structure. It's for people processing scans, handwritten records or documents whose tables and forms need to survive extraction.

Unstructured is a document processing library for developers preparing files for LLM applications. It turns text documents and images into structured output that applications can work with.

CodeFuse ChatBot is a self-hosted AI assistant for developers and DevOps teams who want answers grounded in their own code repositories and documentation. It combines repository analysis with agents that can use tools and execute code in a sandbox, covering work across design, coding, testing, deployment and operations.

MemSearch gives coding assistants searchable project history across sessions. It's for developers who want to recover earlier fixes, revisit design decisions or continue work in another agent.

JamAI Base is an AI backend for developers building chatbots and applications that use their own documents and structured data. Its spreadsheet-like interface brings prompts, generated data and LLM workflows into tables.

RankLLM is a Python toolkit for search researchers and developers who want to compare how AI models reorder retrieved documents by relevance. Its main focus is listwise reranking, where a model considers several candidates together. It supports local inference and optional cloud APIs, and it's open source under the Apache 2.0 license.

Semantica is a Python toolkit for teams building AI systems that need shared context, traceable decisions and policy checks. It connects facts and decisions to their evidence in queryable knowledge graphs.

Langroid is a Python framework for developers building applications where AI agents divide up a task. Each agent can maintain its own conversation and use documents or tools, then delegate work to other agents.

Embedding Atlas helps researchers and developers inspect embedding datasets and the records behind them. It makes clusters, outliers, and relationships visible alongside the data's metadata.

ColBERT is a self-hosted neural search library for developers building document search or retrieval for AI applications. It finds relevant passages in your own text collections and stores their search representations on disk. The Python project is open source under the MIT license.

KubeAI is a Kubernetes operator for teams serving AI models on their own infrastructure or cloud clusters. It manages model servers and adjusts capacity to demand, including starting models from zero running replicas.

BeeAI Framework is for developers building AI agents and systems where several agents work together. It lets developers enforce rules on agent behavior while retaining the model's ability to reason.

Crawl4AI is a web crawler and scraper for developers feeding websites into AI agents, RAG systems and data pipelines. It can extract useful page content without requiring an LLM.

ColGREP helps developers and coding agents find relevant code even when they don't know the exact words to search for. It runs locally, while NextPlaid, the search engine behind it, can also serve other retrieval applications on a self-hosted server.

DataFlow is a Python system for researchers and teams preparing data for LLM training or RAG. It turns noisy PDFs, plain text, and question-answer datasets into data suited to a specific domain or training task.

Sentence Transformers is a Python library for developers building search and document retrieval on their own hardware. It helps match content by meaning and rank results against a query.

Qwen-Agent is a Python framework for developers building assistants that use Qwen models to read files and carry out tasks with tools. It provides ready-made agent components and example applications for custom assistants and browser assistance.

Lance is an open source data format for teams building search engines, feature stores and machine learning training pipelines on object storage. It keeps images, video, audio, text and embeddings alongside tabular data, so those workloads can share a dataset. The project uses the Apache 2.0 license.

Spring AI is a Java framework for developers adding AI features to Spring applications. It connects models with application data and APIs for tasks such as answering questions about documents.

Claude Context adds repository search to AI coding assistants so they can retrieve relevant code without loading whole directories into their context. It's for developers working with large codebases that an assistant needs to understand.

Skill Seekers prepares documentation and code for AI assistants. It's for developers and teams who want reusable skills, coding context or retrieval-augmented generation (RAG) data without preparing the same sources separately for each tool.

OpenViking is a self-hosted context database for developers who want AI agents to retain preferences and task experience across sessions. Its open source server runs on your own machine or server, including Docker deployments. It needs embedding and vision-language models, with support for local Ollama and cloud providers such as OpenAI and Volcengine; self-hosting the database alone doesn't keep model requests local.

LangChain is an open-source framework for developers building AI agents and LLM applications, including apps that run models locally through Ollama. Its Python ChatOllama integration connects to open-weight models such as gpt-oss. You write the application in code, while Ollama handles local inference.

chromem-go is a vector database for Go developers adding semantic search or retrieval augmented generation (RAG) to an application. It runs inside the app, so it doesn't need a separate database server.

Vearch is a self-hosted, distributed vector database for developers building AI applications that search embedding vectors. It combines similarity search with filters on stored fields, so an application can retrieve related content while applying criteria beyond vector similarity. It's open source under Apache 2.0 and can run on your servers through Docker, Docker Compose, or Kubernetes.

Top2Vec is an open-source Python library for researchers and developers who want to discover themes in text collections and search documents by meaning. It determines the number of topics automatically and connects topics, documents and words through a shared embedding space. The license is BSD-3-Clause.

FlashRank is a Python library that runs reranking models locally on CPU, without Torch or Transformers. It's for developers building search systems or retrieval-augmented generation (RAG) apps who want to reorder retrieved passages by relevance before passing them to an LLM.