On the Faithfulness of Conditional Probability Estimates: A Multi-objective Approach

Estimating conditional probabilities from data samples is the building block of many machine learning algorithms. However, these estimates often do not reflect the true underlying distribution. With ML systems becoming more ubiquitous in high-risk decision making, it is critical to investigate how the predictions made by the algorithm can misrepresent the actual data distribution. To this end, we explore the notion of underestimation bias that captures the extent to which a learned model predictions deviate from the true distribution. Since fixing underestimation bias may come at the cost of accuracy, we propose a multi-objective optimization strategy to obtain a diverse set of models with members representing different accuracy/faithfulness tradeoffs. We empirically evaluate our framework on two synthetic and twelve real-world datasets. We show that our framework can address underestimation bias while still maintaining adequate overall generalization accuracy.