Open-Source Training and RLHF Frameworks

Reef is self-hosted infrastructure for developers who want AI agents to learn from their interactions. It connects live requests and feedback to changes in model weights or the prompts, rules, and skills that guide an agent.

Opacus is a Python library for training PyTorch models with differential privacy. It's for machine learning practitioners adapting existing training code and researchers experimenting with private training methods.

SecretFlow is an open-source Python framework for teams that need to analyze data or train machine learning models across separately held datasets. It brings several privacy-preserving computing methods into one framework, so applications can combine them according to their security assumptions and performance needs.

PaddleNLP is a Python toolkit for developers training and deploying language models on their own infrastructure. Built on PaddlePaddle, it covers pretraining, fine-tuning, model compression and inference across several accelerator families.

SpeechBrain is a toolkit for researchers and developers building speech assistants, transcription systems and conversational AI. It provides pretrained models and training recipes for adapting speech, audio and text models to your own data.

InterpretML is an open-source Python library for data scientists who need to explain machine learning predictions or build models whose decisions they can inspect directly. It runs locally on Windows, macOS and Linux under the MIT license. Its tools support model debugging and checks for potential bias, including work in healthcare and finance.

NVIDIA cuML is a Python machine learning library for data scientists whose scikit-learn, UMAP or HDBSCAN workloads take too long on a CPU. It runs on NVIDIA GPUs in Linux or WSL 2, on a workstation or across a cluster. The project is open source under Apache 2.0.

DeepLabCut is an open-source toolbox for researchers who want to track animal movement and behavior on their own computers without attaching physical markers. It estimates the positions of visible features you choose, with support for animals, humans and objects rather than a fixed set of species or body parts.

ONNX Runtime is an AI inference and training engine for developers adding models to apps and services. It supports deployment on a user's device or a server, including applications that need the same model format across platforms.

AReaL is open-source reinforcement learning infrastructure for researchers and developers training LLMs and AI agents on their own compute. Its asynchronous training approach targets large reasoning models and agent applications, with support for single-node runs and distributed training. It uses the Apache 2.0 license.

Candle is a Rust machine learning framework for developers embedding local AI in applications or deploying models on their own servers. It supports inference and training, including small CPU deployments for serverless applications.

OpenPipe ART is a Python framework for developers who want to train multi-step AI agents on the tasks their applications actually perform. It uses reinforcement learning through GRPO (Group Relative Policy Optimization) to improve model behavior based on experience and task rewards. It's open source under Apache 2.0.

FLAML is a Python AutoML library for data scientists and developers who want to reduce manual model selection and tuning. Its search accounts for computational cost when choosing which configurations to evaluate.

spaCy is a Python library for developers and data scientists who need to extract structured information from text in their own applications. It runs on macOS, Windows and Linux, with model packages that become part of your application. It's free and open source under the MIT license.

BasicSR is an image and video restoration toolkit for researchers and developers building software to recover detail or reduce visual defects. It provides restoration models and pipelines for training, testing and inference.

PyTorch Lightning is a Python framework for researchers and developers pretraining or fine-tuning AI models. It handles recurring training infrastructure while leaving the model logic in your PyTorch code.

NeMo Automodel is a Python library for researchers and developers who want to train or adapt AI models on their own GPU workstations or clusters. Its PyTorch-based training works with Hugging Face models without converting checkpoints.

Open Instruct is AllenAI's toolkit for researchers and developers adapting pretrained language models with public datasets. It includes the training recipes behind TÜLU and downloadable checkpoints for comparing training stages.

Nerfstudio is a 3D reconstruction framework for researchers and developers turning photos or phone video into scenes they can view from new angles. It supports model training, testing and inspection within one Python framework.

icefall is a speech model training toolkit for developers and researchers building recognition systems on their own hardware. Its Python recipes use k2 and lhotse to train, fine-tune, and evaluate models across speech datasets.

Amphion is an audio generation toolkit for researchers and engineers developing speech and singing systems. It brings model implementations, dataset preparation and evaluation into one research codebase.

LLaMA-Factory is a local model training platform for developers and researchers who want to adapt pretrained models to their own tasks. Its web UI lets you fine-tune models without writing code.

torchtune is a Python library for developers and researchers who want to adapt LLMs on their own hardware with editable PyTorch training code. The project is no longer actively maintained.

TorchGeo is an open source Python library for machine learning researchers and remote sensing specialists working with satellite and aerial imagery. It brings geographic datasets and pretrained models into PyTorch training workflows. The MIT license allows you to use and adapt it for your own projects.

TorchAO is a PyTorch library for developers who want to train and run models on their own hardware with less memory. It supports language models and image or video generation, and works with torch.compile and FSDP2 across most Hugging Face PyTorch models.

Fed-BioMed lets hospitals and researchers train AI models across institutions while patient data stays on each institution's own infrastructure. It's a self-hosted platform for biomedical research that needs access to data across multiple sites but faces restrictions on sharing clinical records.

nanochat is a toolkit for researchers and developers who want to train an LLM and inspect the pipeline themselves. Its small Python codebase covers the stages between preparing a tokenizer and chatting with the resulting model.

PyTorch Forecasting is a Python library for developers and researchers who want to predict time series with neural networks on their own hardware. It combines ready-made forecasting architectures with data preparation, training, and evaluation tools. It's open source under the MIT license.

nnU-Net is an open-source framework for researchers who need to segment biomedical images using labeled training data. It examines each dataset and chooses suitable U-Net configurations, reducing the amount of pipeline design needed for a new imaging task.

DistillKit is a Python toolkit for developers and researchers training smaller language models to learn from larger ones. It addresses the memory and storage demands of distillation, particularly when you want to reuse one teacher's outputs across several student models.

Hivemind is a Python library for researchers and teams that want to train a shared neural network across computers owned by universities, companies or volunteers. It uses PyTorch and connects participants over the internet, so contributors can pool their machines for a training job without a central master node.

PaddleSpeech is a speech and audio toolkit for developers and researchers building applications on their own machines or servers. Built on PaddlePaddle, it supports pretrained models alongside training and evaluation workflows.

Flair is a free Python library for developers and researchers who need to extract information from text or train their own natural language processing models. It combines pretrained text analysis models with tools for building word and document embeddings, all within a framework built directly on PyTorch.

Dopamine is a reinforcement learning framework for researchers who want to modify agents and test algorithm ideas. Its compact codebase focuses on a limited set of established algorithms and supports experiments on your own hardware.