MTEB
An open-source Python toolkit for evaluating embedding models and retrieval systems, with Sentence Transformers support and an interactive leaderboard.
MTEB is an Apache 2.0 Python toolkit for evaluating embedding models and retrieval systems. It runs evaluations through Python or a command-line interface and publishes an interactive leaderboard.
You can use supported models, including Sentence Transformers, or define a custom model. Select tasks, benchmarks and dataset splits to focus an evaluation on the workload you need to compare. The project covers multilingual and multimodal embeddings.
Evaluation runs support result caching. You can also load existing results for analysis and contribute models, datasets or benchmarks. Benchmark citation metadata and task tables help document research results.
