SkyRL
An open source LLM training library that runs on your own hardware, supports the Tinker API, and includes environments for training tool-using agents.
SkyRL is a Python library for researchers and developers who want to train LLMs with reinforcement learning on their own hardware. Its Tinker API support lets you run existing Tinker training scripts against a server you control without changing the scripts. It's open source under Apache 2.0.
The library brings training and inference into a shared engine, with modular components you can extend for custom tasks and algorithms. It supports GRPO and DAPO, asynchronous training pipelines, and supervised fine-tuning. Docker and source deployments are available, while a Megatron backend supports large-scale training with five-dimensional parallelism.
For agent training, SkyRL includes an agent layer aimed at tasks that require many steps, plus a collection of tool-use environments built around the Gymnasium API. These cover math, coding, search and SQL. Examples include agents that query databases across multiple turns and use search tools, and the Harbor integration supports terminal-use agent training.
The stack suits teams that need to change both the training method and the environment an agent learns in. You can build custom environments and algorithms within the same library rather than limit experiments to its supplied recipes. SkyRL also provides dataset preparation guidance, validated Tinker cookbook recipes, and support for checkpointing and training observability. Local training requires the documented GPU, CUDA and Ray environment; support varies by model and backend.
