Tianshou

Tianshou is a Python reinforcement learning library built on PyTorch and Gymnasium. It's for researchers developing learning algorithms and developers training agents in custom environments. Its two API levels let you choose between a high-level interface for applying existing algorithms and modular interfaces for changing how an algorithm learns.

The library covers online learning with on-policy and off-policy algorithms, plus offline reinforcement learning from existing data. Implementations include DQN and Rainbow DQN, policy methods such as PPO and TRPO, and continuous-control algorithms including TD3 and SAC. For learning from recorded experience or demonstrations, it includes CQL, BCQ, behavior cloning through TD3BC, and imitation learning methods such as GAIL.

Its design separates learning algorithms from policies, training, and environment interactions. That separation gives algorithm researchers room to experiment without having to rebuild the surrounding training process. Replay buffers include prioritized experience replay and hindsight experience replay; intrinsic curiosity modules support exploration.

Tianshou is open source under the MIT license. Its tests exercise full agent training procedures and check that implemented algorithms reach consistent performance within bounded training runs. The project also provides Atari and MuJoCo benchmark examples, alongside tutorials and documentation for algorithm and trainer parameters. Install from PyPI or conda-forge with Python 3.11 or later. Version 2 revises the procedural API and is not backward compatible with version 1.