Optimum

Optimum is a collection of Python packages for developers adapting Hugging Face models to their deployment hardware. It extends Transformers, Diffusers, TIMM and Sentence Transformers with tools for faster inference and training.

• Model export and optimization: Export models to ONNX, apply quantization and optimize model graphs, then run them with ONNX Runtime through the separate optimum-onnx package.
• Hardware acceleration: Target Intel hardware with OpenVINO or NVIDIA hardware with TensorRT-LLM; dedicated packages also support Intel Gaudi and FuriosaAI WARBOY.
• On-device inference: Export Transformers models to ExecuTorch for mobile and edge deployment within the PyTorch ecosystem.
• Accelerated training: Use training integrations for Intel Gaudi, AWS Trainium and ONNX Runtime, alongside packages for Google TPUs and AWS Inferentia.

Optimum is a developer toolkit, with hardware-specific packages for local, edge and cloud deployments. AWS Trainium and Inferentia integrations run on AWS infrastructure. Developers who need custom model optimizations can create and combine PyTorch Transformers graph changes with Torch FX. The project is open source under Apache 2.0.