Promoting Fairness through Hyperparameter Optimization
TL
TL;DR
This work proposes and evaluates fairness-aware variants of three popular HO algorithms: Fair Random Search, Fair TPE, and Fairband, and shows that, without extra training cost, it is feasible to find models with 111% mean fairness increase and just 6% decrease in performance when compared with fairness-blind HO.
Abstract
Considerable research effort has been guided towards algorithmic fairness but real-world adoption of bias reduction techniques is still scarce. Existing methods are either metric-or model-specific, require access to sensitive attributes at inference time, or carry high development or deployment costs. This work explores the unfairness that emerges when optimizing ML models solely for predictive performance, and how to mitigate it with a simple and easily deployed intervention: fairness-aware hyperparameter optimization (HO). We propose and evaluate fairness-aware variants of three popular HO algorithms: Fair Random Search, Fair TPE, and Fairband. We validate our approach on a real-world bank account opening fraud case-study, as well as on three datasets from the fairness literature. Results show that, without extra training cost, it is feasible to find models with 111% mean fairness increase and just 6% decrease in performance when compared with fairness-blind HO.1
References50 items
1
Bias and Fairness
2
A Bandit-Based Algorithm for Fairness-Aware Hyperparameter Optimization
3
Dealing with Bias and Fairness in Data Science Systems: A Practical Hands-on Tutorial
4
Fair Bayesian Optimization
5