PyTerrier

An open-source Python framework for search and RAG pipelines, with retrieval evaluation on standard datasets. Runs on Linux, Windows and macOS under MPL-2.0.

PyTerrier is a Python framework for researchers and developers building search systems or the retrieval part of a RAG application. It builds on Terrier and lets you combine retrieval and reranking components into pipelines, then compare their search effectiveness on standard datasets.

It supports sparse, learned sparse and dense indexing and retrieval, along with query expansion and learning-to-rank. A pipeline can pair Terrier's BM25 retriever with a MonoT5 neural reranker. In notebooks, you can also visualize the components and how they connect, which helps when examining a search experiment.

The extension ecosystem adds several concrete choices for retrieval and ranking:

PyTerrier_DR supports single-representation dense retrieval; PyTerrier_ColBERT2 supports multiple-representation retrieval and neural reranking.
PyTerrier_T5 provides monoT5 and duoT5 reranking, while PyTerrier_GenRank supports generative listwise reranking with RankVicuna and RankZephyr.
PyTerrier_RAG adds retrieval augmented generation and LLM access. PISA, doc2query and SPLADE extensions cover in-memory retrieval and neural augmented indexing.

For evaluation, PyTerrier can compare pipelines using datasets available through ir_datasets, including TREC Genomics. This makes it useful for testing retrieval choices against a shared benchmark before choosing a pipeline for an application.

PyTerrier is open source under the Mozilla Public License 2.0. It runs on Linux, Windows and macOS, and its example notebooks also run on Google Colab.