Infinity Embeddings

Infinity Embeddings is a self-hosted model server for developers building semantic search and retrieval-augmented generation applications. It runs retrieval models on your own hardware and can serve several models together.

• Text search and reranking: Generate embeddings for search, clustering and recommendations, or score documents against a query to sort results by relevance.
• Model choice: Use Hugging Face models or local model files, including jinaai/jina-embeddings-v3, BAAI/bge-m3 and mixedbread-ai/mxbai-rerank-xsmall-v1; bge-m3 support excludes sparse embeddings.
• Image and audio search: Use CLIP to match images with text and CLAP to match audio with text.
• Text classification: Run models for sentiment analysis, emotion detection and other classification tasks alongside retrieval models.
• Application connections: Access models through a REST API aligned with OpenAI's API specifications or an asynchronous Python interface, with integrations including LangChain, Ragflow and KubeAI.

Infinity is open source under the MIT license and supports Docker deployment. Inference runs on the host machine using CPUs, NVIDIA CUDA, AMD ROCm, Apple MPS or AWS Inferentia hardware. Supported backends include PyTorch, ONNX/TensorRT and CTranslate2. Dynamic batching groups requests for processing under load.