ColBERT
An open-source neural search library for your own text collections, with BERT-based retrieval, CPU search and GPU indexing under the MIT license.
ColBERT is a self-hosted neural search library for developers building document search or retrieval for AI applications. It finds relevant passages in your own text collections and stores their search representations on disk. The Python project is open source under the MIT license.
Its distinctive approach is called late interaction. Instead of reducing a whole passage to one vector, ColBERT keeps representations of individual tokens and compares them with the query's tokens. This preserves finer contextual matches while allowing search across large collections. Passage representations are computed ahead of time, so each query doesn't require processing every document again.
ColBERT includes the PLAID retrieval engine and residual compression for its indexes. It retrieves ranked passages directly from the full collection, and you can adjust the balance between search speed and result quality. A pretrained checkpoint trained on MS MARCO Passage Ranking is available, along with support for training your own retrieval model.
The library uses PyTorch and Hugging Face Transformers. Search can run on a CPU, but training and indexing require a GPU. Its Python API covers indexing, retrieval and training; a lightweight local server returns ranked search results as JSON.
RAGatouille provides an application-oriented way to work with ColBERT. DSPy can combine its retriever with language models, and the library accepts TSV text collections, queries and ranked results.
