Accelerate
A PyTorch library for local or distributed training and inference, with CPU, GPU and TPU support and automatic mixed precision.
Accelerate is a Python library for developers and researchers who write their own PyTorch training loops. It handles hardware and distributed execution so they can keep control of how their models train.
- Local and distributed workloads: Use the same PyTorch code on a local machine or across multiple machines for training and inference.
- Hardware support: Run on CPUs, GPUs or TPUs, including multiple CPUs or GPUs across machines, with automatic device placement.
- Mixed precision: Train with FP16, BF16 or FP8; FP8 support uses Transformer Engine or MS-AMP.
- Notebook training: Run distributed training from notebooks, including Colab and Kaggle notebooks with a TPU backend.
Accelerate is open source under Apache 2.0 and runs in your chosen local or cluster environment. Colab and Kaggle notebook workloads run on those hosted services. Its command-line launcher is optional, and you'll still need to write the training loop yourself.
Logotyper
