Cog
Package machine learning models as Docker containers with HTTP APIs for deployment on your own servers or Replicate.
Cog packages machine learning models for researchers and developers serving them on their own infrastructure. It handles the container environment and inference API around a Python model.
- Model packaging: Build Docker images with model dependencies, caching and support for specific Python versions.
- GPU compatibility: Coordinate compatible combinations of CUDA, cuDNN, PyTorch, TensorFlow and Python for NVIDIA GPU workloads.
- Inference APIs: Serve each model through an HTTP API with an OpenAPI schema and validation based on its Python input and output types.
- Deployment choices: Run models locally or on your own Docker infrastructure, with Replicate available as an optional hosted service.
The Apache 2.0 tool requires Docker and supports macOS, Linux and Windows 11 through WSL2. GPU execution requires suitable NVIDIA hardware and drivers. Model weights have their own licenses, and models that call external services retain those network dependencies.
Logotyper
