NeMo RL
An open-source model post-training library for your own GPUs, with reinforcement learning, fine-tuning and PyTorch or Megatron Core backends.
NeMo RL is NVIDIA's Python library for researchers and engineering teams adapting language and multimodal models on their own hardware. It supports experiments on a GPU workstation and distributed training across multiple nodes.
- Post-training methods: Train with rewards using GRPO or PPO, apply supervised fine-tuning or DPO, and train reward models.
- Agent and multimodal training: Train multi-turn tool use and game interactions with custom reward environments, plus vision-language and audio tasks.
- Distillation: Transfer behavior from teacher models through on-policy and multi-teacher training, or use off-policy distillation when teacher and student tokenizers differ.
- Training efficiency: Use LoRA to train fewer model parameters, sequence packing to use training capacity efficiently, and FP8 training through Megatron Core.
- Evaluation: Assess trained models with built-in evaluation datasets or custom evaluation pipelines.
Training uses PyTorch-native DTensor or Megatron Core, with Ray coordinating distributed resources. Generation backends include vLLM, SGLang and Megatron inference. Hugging Face integration supports model loading and checkpoint export; supported model families include Qwen and Nemotron.
NeMo RL is open source under Apache 2.0. It runs on local workstations or your own training infrastructure, with Docker support and deployment across Slurm or Kubernetes clusters.
