MAPIE: an open-source library for distribution-free uncertainty quantification
TL
TL;DR
MAPIE (Model Agnostic Prediction Interval Estimator), an open-source Python library that quantifies the uncertainties of ML models for single-output regression and multi-class classification tasks, is introduced.
Abstract
Estimating uncertainties associated with the predictions of Machine Learning (ML) models is of crucial importance to assess their robustness and predictive power. In this submission, we introduce MAPIE (Model Agnostic Prediction Interval Estimator), an open-source Python library that quantifies the uncertainties of ML models for single-output regression and multi-class classification tasks. MAPIE implements conformal prediction methods, allowing the user to easily compute uncertainties with strong theoretical guarantees on the marginal coverages and with mild assumptions on the model or on the underlying data distribution. MAPIE is hosted on scikit-learn-contrib and is fully"scikit-learn-compatible". As such, it accepts any type of regressor or classifier coming with a scikit-learn API. The library is available at: https://github.com/scikit-learn-contrib/MAPIE/.
References19 items
1
Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control
2
Adaptive Conformal Inference Under Distribution Shift
3
Distribution-free uncertainty quantification for classification under label shift
4
Copula-based conformal prediction for Multi-Target Regression
5