torchtune

torchtune is a Python library for developers and researchers who want to adapt LLMs on their own hardware with editable PyTorch training code. The project is no longer actively maintained.

• Supervised fine-tuning: Train with full weight updates or memory-saving LoRA and QLoRA methods on one GPU, multiple GPUs or multiple machines.
• Knowledge distillation: Train smaller models with larger models as teachers, using LoRA or QLoRA on one or multiple devices.
• Preference and reinforcement learning: Apply DPO, PPO or GRPO, with weight-update methods and distributed training support varying by recipe.
• Quantization and evaluation: Prepare models for reduced-precision use with quantization-aware training, and use separate recipes for evaluation, quantization and inference.

Supported model families include Llama, Llama Vision, Gemma, Mistral, Microsoft Phi and Qwen. The model implementations and training recipes use PyTorch directly, so you can modify training behavior for your experiments.

Training runs on your own GPU hardware, with NVIDIA examples including the RTX 4090, A6000 and A100. Memory needs depend on the model and training method; activation checkpointing and CPU offload can reduce GPU memory use. Supervised fine-tuning and full-weight GRPO support multiple machines, while distillation, DPO, PPO and quantization-aware training have narrower distributed support. The library is open source under the BSD 3-Clause license.