AReaL
Open-source reinforcement learning infrastructure for LLMs and agents, with asynchronous training, PyTorch and Megatron backends, and Apache 2.0 licensing.
AReaL is open-source reinforcement learning infrastructure for researchers and developers training LLMs and AI agents on their own compute. Its asynchronous training approach targets large reasoning models and agent applications, with support for single-node runs and distributed training. It uses the Apache 2.0 license.
Agent training is a central use case. AReaL supports custom agent workflows and online reinforcement learning for applications whose internal implementation isn't accessible. Examples cover math, coding, search and customer service tasks, along with visual reasoning. Developers can customize datasets, algorithms and process rewards to fit their training task.
The training methods include PPO, GRPO and related algorithms, with both synchronous and asynchronous execution. It also supports direct preference optimization, supervised fine-tuning, reward modeling and on-policy distillation. Evaluation, checkpointing and metrics tracking help teams assess training results.
Model support includes Qwen2 and Qwen3, Qwen mixture-of-experts models, and vision-language families such as Qwen2.5-VL, Qwen3-VL and Gemma 3. Compatibility depends on the training backend. AReaL works with Megatron, PyTorch FSDP and its PyTorch-native Archon engine, with distributed training capabilities for large models. LoRA training is available through supported backends, and its vision-language training examples include Huawei NPU hardware. Training requires a compatible GPU/CUDA stack or the documented Huawei NPU environment; model support depends on the backend.
