OpenCLIP

OpenCLIP is a Python library for developers and researchers building image-text matching and zero-shot classification applications. It supports pretrained CLIP models and training models on your own hardware.

• Pretrained models: Use a shared model interface for pretrained and untrained models, with checkpoints loaded from local files or downloaded from Hugging Face.
• Custom training: Train with multiple datasets and adjust how often each source contributes examples, rather than letting dataset size alone determine the mix.
• Distributed training: Run training across multiple GPUs or machines, with native support for SLURM clusters and training tested on NVIDIA A100 hardware.
• Text encoders: Use the supported Hugging Face RoBERTa configuration as the language component of a CLIP model.

The library uses PyTorch and accepts local .pt checkpoints and downloaded Hugging Face model files. Local checkpoint loading lets model inference run on your hardware. Its training interface supports larger research workloads as well as single-machine runs.