Dataset and Synthetic Data Tools

Nemotron is NVIDIA's family of AI models for developers building agents that reason, write code and use tools. Teams can evaluate and adapt the models using published weights, training data and training recipes.

VoxelMorph is a Python library for researchers and developers who need to align medical images using learned deformations. It provides registration networks and training components, including unsupervised methods, with PyTorch and TensorFlow backends. The code is open source under the Apache 2.0 license.

DataDreamer is a Python library for researchers and developers who need to create training datasets and adapt LLMs to specific tasks. It connects data preparation and model training in reproducible workflows.

Oumi helps ML researchers and teams build specialized AI models for their own tasks. It brings data preparation, training, evaluation, and deployment into one platform, with reusable recipes and components you can customize.

MONAI is a framework for researchers and developers building AI models for medical images. Its Core library provides medical-specific tools for training and evaluating models within existing PyTorch workflows.

DeepTrack2 is an open-source Python library for researchers building image data pipelines for machine learning and digital microscopy. It brings image generation, manipulation and analysis into a modular framework that runs in your own Python environment. Its focus is scientific imaging, with applications in microscopy, optics and biophysics.

Cleanlab is a Python library for data scientists and ML engineers checking training data for errors. It uses model predictions or feature embeddings to identify problems in text, image, audio and tabular datasets.

Hugging Face Datasets is a Python library for developers and researchers preparing data for AI training and evaluation. It handles local files and datasets from the Hugging Face Hub through the same data-processing tools.

Synthetic Data Vault is a Python library for teams that need synthetic versions of their tabular datasets. It learns patterns from real records and generates data that emulates them. Training and generation run on your own hardware with standard CPUs.

Augmentoolkit turns documents into datasets for training an LLM on a particular subject. It's for people building custom models around material such as research papers or fictional lore.

NVIDIA NeMo Data Designer is a Python framework for developers and data teams creating synthetic datasets with controlled variation and relationships between fields. It can start with an empty dataset or build on your own seed data. The framework is open source under Apache 2.0.

NVIDIA Cosmos provides world models for developers building robots, autonomous vehicles and industrial vision systems. It supports reasoning about physical scenes, generating training data and learning actions before real-world deployment.

DataTrove is a Python library for teams preparing large text datasets, including LLM training corpora. It combines reusable processing blocks with custom cleanup logic, so teams can apply their own rules across large workloads.

Kiln is a desktop workbench for teams building and testing AI applications. Engineers, QA staff and subject matter experts can work with the same tasks and datasets, including contributing ratings and examples without writing code.

LLaVA is a vision-language model project for researchers and developers who want to ask questions about images on their own hardware.

TorchGeo is an open source Python library for machine learning researchers and remote sensing specialists working with satellite and aerial imagery. It brings geographic datasets and pretrained models into PyTorch training workflows. The MIT license allows you to use and adapt it for your own projects.

Argilla is a data annotation and feedback tool for AI engineers and domain experts building training and evaluation datasets. It gives reviewers a shared interface for checking examples and contributing judgments that inform model development.

Curator is a Python library for developers building LLM training datasets or extracting structured records from existing data. It supports pipelines that generate data in stages and process many model requests.

Distilabel is a Python framework for engineers building synthetic datasets and model-generated feedback with local LLM backends such as Ollama, llama.cpp and Hugging Face Transformers. It also connects to cloud services, including OpenAI and Anthropic; requests through those integrations go to the remote provider. Where inference runs depends on the backend you choose.

Ragas is a Python library for developers evaluating LLM applications and retrieval-augmented generation systems. Its non-LLM metrics compute text comparisons in your Python environment without a model, including string similarity, BLEU and ROUGE. Its model-based quickstart uses OpenAI's GPT-4o and an API key, sending evaluation inputs to OpenAI's cloud service. Local text scoring and cloud model judgments are separate evaluation options.

SDMetrics is an open source Python library for people who need to assess synthetic tabular data before using it in software testing or machine learning development. It compares generated datasets with the real data they're meant to resemble. It works with data from any model, so your choice of generator doesn't determine how you evaluate the results.

Data-Juicer is a Python framework for researchers and teams preparing datasets for AI training, evaluation, agents or document retrieval. It processes data on your own machine or across a Ray cluster.

DataFlow is a Python system for researchers and teams preparing data for LLM training or RAG. It turns noisy PDFs, plain text, and question-answer datasets into data suited to a specific domain or training task.

NeMo Curator is an open source Python toolkit for ML engineers and data teams preparing AI training datasets. It provides repeatable curation pipelines that can run on a laptop or across a multi-node Ray cluster.

CAMEL is a Python framework for developers and researchers building AI agent teams with models running locally through Ollama or through remote providers. Its Ollama integration connects agents to a local model service; choosing OpenAI instead sends model requests to OpenAI and requires an API key. The framework is open source under Apache 2.0.

DeepEval is an LLM evaluation framework for developers testing AI agents, RAG pipelines and chatbots. It helps compare prompts and models and catch quality regressions within Python tests or CI/CD.