People underestimate the errors by algorithms for credit scoring and recidivism but tolerate even fewer errors

This study provides the first representative analysis of error estimations and error tolerance in a Western country (Germany) with regards to algorithmic decision-making systems (ADM). We examine people’s expectations about the accuracy of algorithms that predict credit default, recidivism of an offender, suitability of a job applicant, and health behavior. Also, we ask whether expectations about algorithm errors vary between these domains and how they differ from expectations about errors made by human experts. In a nationwide representative study (N=3,086) we find that most respondents underestimated the actual errors made by algorithms and are willing to tolerate even fewer errors than estimated. Error estimates and error tolerance did not differ consistently for predictions made by algorithms or human experts, but people’s living conditions (e.g. unemployment, household income) affect domain-specific tolerance (job suitability, credit defaulting) of misses and false alarms. We conclude that people have unwarranted trust in the competence of ADM systems and evaluate errors in terms of potential personal consequences. Given the general public’s low error tolerance, we further conclude that acceptance of ADM appears to be conditional to strict accuracy requirements.