Conversational User Interfaces: Explanations and Interactivity Positively Influence Advice Taking from Generative Artificial Intelligence

Generative artificial intelligence (GenAI) has surged in popularity with the implementation of conversational user interfaces. The possibility to engage in natural conversations with GenAI thus constitutes a promising countermeasure to algorithm aversion, or the general preference for humans over algorithms, which hinders the effective utilization of often superior algorithmic output. In this study, we experimentally test the influence of explanations and interactivity on advice taking from GenAIs. In a judge-advisor system, 472 participants (313 female, 154 male, 5 diverse; median age 23 years) solved a series of ten estimation tasks with access to pre-generated output from ChatGPT. In the control condition, only the numerical output was provided as advice. Participants in the treatment conditions were additionally provided with, or could request, a detailed explanation of the rationale underlying ChatGPT’s judgments. The weight of advice was positively influenced by both the opportunity to interact with the GenAI and the receipt of an explanation. Moreover, actively requesting an explanation significantly enhanced the positive effect of interactivity compared to trials on which this opportunity was forgone. However, there was no evidence that the weighting of algorithmic advice was influenced by whether an explanation was provided or requested. The inherent explanatory capabilities of GenAI and the opportunity to interactively engage with it independently increase users’ advice taking. This finding underscores the potential of conversational user interfaces to large language models such as ChatGPT to improve individuals’ augmented judgment and decision-making, but also poses a threat to human autonomy in interactions with conversational GenAI systems.