Stable Baselines3

Stable Baselines3 is a Python library for training reinforcement learning agents on your own hardware using PyTorch. It builds on Stable Baselines and gives researchers and developers tested algorithm implementations for comparing approaches or building their own projects. It's free and open source under the MIT license.

The library includes PPO, A2C, DQN and DDPG, with a common interface that reduces the amount of algorithm-specific code you need to learn. Support for custom environments and policies lets you adapt training to your own problem. It also handles dictionary observations and supports multiprocessing across its core algorithms.

You need some RL experience. SB3 suits people who want established implementations as a starting point while retaining control over the environment and the agent's policy. Its syntax follows a scikit-learn-like style, and it works with IPython and notebooks.

TensorBoard support and custom callbacks help you track training and add behavior around it. Integrations with Weights & Biases provide experiment tracking, while Hugging Face supports storing and sharing trained models through an external service. Algorithm performance tests and reports on OpenRL Benchmark give you reference results for comparisons.

The separate RL Baselines3 Zoo training framework adds evaluation, hyperparameter tuning, plots and video recording. It also supplies tuned settings for common environments and algorithms, along with agents trained using those settings. Install with pip using Python 3.10 or later and PyTorch 2.8 or later. Core development focuses on maintenance; experimental additions live in separate SB3-Contrib and SBX projects.