Fair machine learning models for disease prediction: In-depth interviews with key health experts

Abstract Artificial intelligence (AI) and machine learning (ML) pose enormous potential for improving quality of life. It can also generate significant social, cultural and other unintended risks. We aimed to explore fairness concepts that can be applied in ML models for disease prediction from key health experts’ perspectives in an ethnically diverse high-income country. In-depth interviews with key experts in the health sector in Aotearoa New Zealand (NZ) were implemented between July and December 2022. We invited participants who are key leaders in their ethnic communities, including Māori (Indigenous), Pasifika and Asian. The interview questionnaire comprised six sections: (1) Existing attitudes to healthcare allocation; (2) Existing attitudes to data held at the general practitioner (GP) level; (3) Acceptable data to have at the GP level for disease prediction models; (4) Trade-offs for obtaining benefits vs generating unnecessary concern in deploying these models; (5) Reducing bias in risk prediction models; and (6) Including community consensus into disease prediction models for fair outcomes. The study shows that participants were strongly united in the view that ML models should not create or exacerbate inequities in healthcare due to biased data and unfair algorithms. An exploration of fairness concepts showed that carefully selected data types must be considered for predictive modelling and that trade-offs for obtaining benefits versus generating unnecessary concern produced conflicting opinions. The participants expressed high acceptability for using ML models but expressed deep concerns about inequity issues and how these models might affect the most vulnerable communities (such as Māori in middle-ages and above and those living in deprived communities). Our results could help inform the development of ML models that consider social impacts in an ethnically diverse society. Competing Interest Statement The authors have declared no competing interest. Funding Statement This work was funded by the Royal Society Te Apārangi. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the report. Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: The study was approved by University of Otago ethics approval processes, reference number HD20/012 and D22/101. There were no patients directly involved in this study. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Data Availability All data are confidential and not available for sharing.