Optuna
A Python hyperparameter optimization framework under the MIT license, with dynamic search spaces, parallel trials and support for PyTorch and scikit-learn.
Optuna is a Python framework for machine learning developers and researchers who want to automate the search for model settings. Its search spaces can change with the choices made during a trial, so it can compare model architectures or classifiers as well as tune individual parameters. It's open source under the MIT license and can run on your own hardware as part of a Python workload.
Search spaces use ordinary Python conditionals and loops. That flexibility matters when parameters depend on each other: a search can choose a classifier, then explore only the settings relevant to that classifier. For neural networks, it can vary the number of layers and the size of each layer within the same search.
Optuna's optimization algorithms select parameter values to test, while pruning stops unpromising trials early to reduce time spent on poor candidates. Searches can run across multiple threads or processes, and distributed optimization supports larger workloads. It also handles objectives that need to be minimized or maximized, with examples covering multiple objectives and constraints.
The framework works with PyTorch, TensorFlow, Keras, scikit-learn, LightGBM and XGBoost, without tying the search to one training library. Built-in plots show optimization history. The separate Optuna Dashboard provides a browser interface with graphs and tables for reviewing results and hyperparameter importance.
