DeepSpeed

DeepSpeed is an open-source PyTorch library for developers and researchers training or running large AI models on their own hardware or compute clusters. It helps address memory limits and distribute workloads across devices.

• Training memory: ZeRO reduces memory requirements when training large models.
• Distributed training: Data, model and pipeline parallelism let teams spread training across multiple devices.
• Specialized workloads: DeepSpeed-MoE supports mixture-of-experts training and inference, while Arctic Long Sequence Training supports training with very long sequences.
• Transformer inference: DeepSpeed includes optimizations for running trained transformer models.

Integrations include Hugging Face Transformers and Accelerate, Lightning and MosaicML. Teams can use these frameworks with DeepSpeed for distributed training; BLOOM and GPT-NeoX are examples of models trained with the library.

DeepSpeed uses the Apache 2.0 license and requires PyTorch. It supports NVIDIA and AMD GPUs, with CUDA or ROCm for GPU extensions. Many training and inference features work on Windows, but asynchronous I/O and GPU Direct Storage aren't supported there.