SDMetrics

SDMetrics is an open source Python library for people who need to assess synthetic tabular data before using it in software testing or machine learning development. It compares generated datasets with the real data they're meant to resemble. It works with data from any model, so your choice of generator doesn't determine how you evaluate the results.

The library distinguishes statistical similarity from usefulness for a particular task. A synthetic dataset can resemble the original without delivering the same value in a downstream project, and SDMetrics includes measures for both. Its data metrics cover quality, privacy and basic diagnostics; its ROI metrics assess the benefit of using synthetic data for the intended task. Those results don't necessarily correlate.

Reports provide visual results that you can examine in detail, rather than relying only on an overall score. You can save reports and share them with your team, or apply individual metrics when you need to assess a specific property. This gives teams a common way to compare datasets produced by different generators.

SDMetrics is a standalone library within DataCebo's Synthetic Data Vault project. You don't need to generate data with SDV to evaluate it here. DataCebo maintains the library, and its MIT license permits use and modification.