Liger Kernel

A Python library of fused kernels for LLM training and fine-tuning, with Hugging Face model patching and support for distributed training.

Liger Kernel is a Python library for developers training or fine-tuning LLMs on their own hardware. Fused kernels reduce memory use during model operations and post-training.

Model operations: Replace operations in compatible Hugging Face models or use individual kernels, including normalization, rotary embeddings and cross entropy, in custom models.
Post-training losses: Use optimized DPO, ORPO and SimPO losses for alignment tasks.
Distributed training: Combine the kernels with Flash Attention, PyTorch FSDP, DeepSpeed or DDP.
Trainer integrations: Work with Axolotl, LLaMA-Factory, SFTTrainer and Hugging Face Trainer.

The BSD 2-Clause library runs within your own training environment with PyTorch and Triton. Backend support includes NVIDIA CUDA, AMD ROCm and Ascend NPU, with coverage varying by operation and hardware. Memory savings and throughput depend on the model and training configuration.

LLM Libraries and SDKs LLM Fine-Tuning Distributed execution