Open Instruct
An open-source LLM post-training toolkit with Docker support, preference training and reinforcement learning recipes for Llama and OLMo.
Open Instruct is AllenAI's toolkit for researchers and developers adapting pretrained language models with public datasets. It includes the training recipes behind TÜLU and downloadable checkpoints for comparing training stages.
- Instruction fine-tuning: Train models on example requests and responses using datasets in a common format, with support for LoRA and QLoRA.
- Preference and reward training: Use direct preference optimization (DPO) to train on preferred responses, or reinforcement learning with verifiable rewards (RLVR), including GRPO recipes.
- Model support: Work with Llama 3.1 and OLMo 2 recipes; the more GPU-efficient OLMo-core supervised fine-tuning implementation supports OLMo, OLMoE and Qwen3.
- Dataset overlap checks: Measure overlap between instruction-training data and evaluation datasets to investigate contamination.
The Python code can run on your own infrastructure and supports Docker. Hugging Face hosts downloads of intermediate and final checkpoints, plus reward models; a separate hosted demo lets you try trained models.
The code is open source under Apache 2.0. Released models have separate terms, including base-model licenses and the AI2 ImpACT license for TÜLU 2. This is a research codebase, so backward compatibility isn't guaranteed. Its built-in evaluation code is unmaintained; the project recommends OLMES for evaluation.
