Analysis of the Reliability and Efficiency of Information Extraction Using AI-Based Chatbot: The More for Less Paradox

This paper addresses the following problem of information extraction using an AI-powered chatbot. An AI chatbot processes a natural language conversation with a human user using semantic search and human-chatbot interaction to extract the most relevant and contextually appropriate information from available databases. During a natural language conversation between a human and an AI, the human user can modify and refine queries until he/she is satisfied with the chatbot’s output. The reliability of an AI chatbot is defined as its ability to understand user queries and provide correct answers; in the current study, it is measured by the frequency (probability) of correct answers. The search efficiency of a chatbot indicates how accurate and relevant the information returned by the chatbot is; in this work, it is measured by the satisfaction level that the human user receives for a correct answer. We uncover a counterintuitive relationship between AI chatbot reliability and search efficiency: we demonstrate that, under fairly general conditions, a less reliable AI chatbot can have a higher expected search efficiency than a more reliable chatbot. This phenomenon aligns with a family of “more-for-less” paradoxes observed in various complex systems. Finally, we discuss the underlying mechanism of this paradox.