Fairlearn

Fairlearn is an open-source Python toolkit for data scientists who need to examine how machine learning models affect different groups of people. It combines fairness assessment metrics with algorithms for reducing observed disparities, under the MIT license.

Its assessment tools help identify groups a model negatively affects and compare models on both fairness and accuracy. The relevant groups depend on the application and are chosen by the data scientist. In lending, for example, an assessment can examine whether a default prediction model makes more errors for applicants of one sex than another.

Mitigation algorithms address several fairness definitions across different AI tasks. They let practitioners assess possible ways to reduce unfairness and weigh the trade-offs for their use case. Fairness metrics can conflict, so satisfying one measure doesn't mean a system meets every definition of fairness.

The project's approach centers on harms to people: withholding opportunities or resources, as in hiring and school admissions, and differences in how well a service works for different groups. It also treats fairness as a social and technical problem. Quantitative measures don't capture concerns such as justice and due process, and algorithms are only one part of addressing those concerns.

Alongside the package, Fairlearn provides guides, use cases, and Jupyter notebook examples, including an assessment of disparities in credit-card default predictions.