Open-Source RAG Frameworks

LangChain4j is a Java library for building chatbots, assistants and AI agents that use local models through Ollama. It runs within JVM applications, while Ollama handles model inference on your hardware using a CPU or GPU. You can also connect to OpenAI or Google Vertex AI, in which case model requests go to the remote provider.

PocketFlow is a Python framework for developers building AI agents and workflows with their own model connections. Its utility examples include an Ollama function for local LLM calls. Developers supply that function rather than using a built-in model wrapper. Cloud API examples send model requests to their providers. FastAPI examples show chat applications with streamed responses and background jobs.

PyTerrier is a Python framework for researchers and developers building search systems or the retrieval part of a RAG application. It builds on Terrier and lets you combine retrieval and reranking components into pipelines, then compare their search effectiveness on standard datasets.

Memvid gives AI agents persistent memory in a portable file. It's for developers who need agents to recall conversations or search documents without running a separate database server.

LLPhant is a PHP framework for developers adding AI capabilities to Symfony and Laravel applications. It supports local model services and cloud providers, so you can choose where model requests run.

Bisheng is a self-hosted AI application platform for teams working with business documents and processes. It supports tasks such as document review, support ticket assistance and fixed-layout report generation, including workflows that need human judgment during execution.

M Flow is a self-hosted memory engine for developers whose AI agents need to recall past conversations and work preferences. It runs in Python or Docker and supports Ollama for local model use, alongside providers such as OpenAI and Anthropic. Choosing a cloud provider sends model requests to that service.

Neo4j LLM Graph Builder converts documents and web content into knowledge graphs for people building searchable collections or AI applications with Neo4j. It uses an LLM to identify entities, relationships, and their properties.

Quivr is a content search and monitoring engine for developers building applications around incoming material and topics they want to follow over time. It's at an evaluation stage, and the project advises against production use because its API may change without notice.

Dify is a visual builder for teams creating AI agents and applications that use their own data and tools. Its canvas makes application logic visible so teams can inspect and test how a task proceeds.

llmware is a Python framework for developers building applications that answer questions using their own documents or carry out multi-step AI agent workflows. It focuses on small, specialized models that can run on laptops, edge devices, and self-hosted servers.

PageIndex is a document retrieval engine for developers building question answering over long PDFs, including financial reports, legal documents and technical manuals. It uses an LLM to select relevant sections from a document tree without a vector database or document chunking.

GraphRAG is an open-source Python indexing and retrieval pipeline for developers building question answering over their own document collections. It can connect to local models through Ollama proxies, while OpenAI models are its default supported choice. With a remote model provider, text sent for model processing leaves your machine.

DB-GPT is a self-hosted AI data assistant for teams analyzing business data and developers building data applications. It connects plain-language questions to database queries and code-based analysis.

RAGFlow helps teams build AI agents that answer questions using their own documents. It extracts content from complex files and retrieves supporting material that readers can check.

JamAI Base is an AI backend for developers building chatbots and applications that use their own documents and structured data. Its spreadsheet-like interface brings prompts, generated data and LLM workflows into tables.

Semantica is a Python toolkit for teams building AI systems that need shared context, traceable decisions and policy checks. It connects facts and decisions to their evidence in queryable knowledge graphs.

Langroid is a Python framework for developers building applications where AI agents divide up a task. Each agent can maintain its own conversation and use documents or tools, then delegate work to other agents.

BeeAI Framework is for developers building AI agents and systems where several agents work together. It lets developers enforce rules on agent behavior while retaining the model's ability to reason.

Qwen-Agent is a Python framework for developers building assistants that use Qwen models to read files and carry out tasks with tools. It provides ready-made agent components and example applications for custom assistants and browser assistance.

Spring AI is a Java framework for developers adding AI features to Spring applications. It connects models with application data and APIs for tasks such as answering questions about documents.

Claude Context adds repository search to AI coding assistants so they can retrieve relevant code without loading whole directories into their context. It's for developers working with large codebases that an assistant needs to understand.

Skill Seekers prepares documentation and code for AI assistants. It's for developers and teams who want reusable skills, coding context or retrieval-augmented generation (RAG) data without preparing the same sources separately for each tool.

OpenViking is a self-hosted context database for developers who want AI agents to retain preferences and task experience across sessions. Its open source server runs on your own machine or server, including Docker deployments. It needs embedding and vision-language models, with support for local Ollama and cloud providers such as OpenAI and Volcengine; self-hosting the database alone doesn't keep model requests local.

LangChain is an open-source framework for developers building AI agents and LLM applications, including apps that run models locally through Ollama. Its Python ChatOllama integration connects to open-weight models such as gpt-oss. You write the application in code, while Ollama handles local inference.

EmbedAnything is a local embedding pipeline for developers building semantic search and retrieval-augmented generation (RAG) applications. It turns documents and media into vectors for indexing in a vector database.

Morphik Core is a retrieval engine for developers building AI applications around documents whose meaning depends on charts, diagrams, or page layout. It brings document search and structured extraction into one platform.

MaxKB is an AI agent platform for organizations building support bots, internal knowledge assistants and business automation. It combines answers based on company documents with workflows that can act through connected tools.

PrivateGPT is a self-hosted API layer for developers building applications around local models. It requires a separate inference server, such as Ollama, llama.cpp, vLLM or LocalAI; it doesn't run models itself.

Ragpi is a self-hosted AI assistant API for teams that want to answer technical questions using their own project material. It lets developers add documentation-based support to a product, website or community.

OneRAG is a self-hosted RAG framework for developers building chatbots that answer questions using their own documents. It combines document retrieval and AI answers, with replaceable components for different models and search needs.

SwarmVault is a self-hosted knowledge base for people organizing research, notes or code documentation for themselves and AI agents. It turns collected sources into a persistent Markdown wiki with connections you can inspect.

OpenRAG is a self-hosted platform for searching documents and asking questions about them through an AI chat interface. It brings Langflow, OpenSearch, and Docling together for developers and teams that want a document knowledge base with control over its retrieval workflows. It's open source under Apache 2.0 and supports self-managed deployment with Docker or Podman.

Genkit is Google's framework for developers building AI agents and chatbots into web and mobile apps. It brings model connections, application workflows and testing tools into one framework.