Assessing the Current Performance of ChatGPT and the Implications in the Academic and Scientific Community: Failures, Misuse, and Challenges

Herein are reported the main pros and cons of using the conversational Artificial Intelligence (AI) called “ChatGPT” as a tool to write essays in the academic and scientific community. Also, selected mathematical and scientific problems were solved to determine the effectiveness of this platform. To this end, a systematic case study was designed and conducted to obtain data generated by the chatbot that was further analyzed. Six different topics were chosen for the essay evaluation, and the introduction section was created using this AI. Meanwhile, six mathematical problems related to integrals, z-transform, Laplace transform, and resolution of ordinary differential equations were solved. Furthermore, a scientific problem associated with the structural properties of material was also solved. After data analysis, we concluded that the conversational platform could quickly create written essays with well-structured words. Nonetheless, in almost all cases, the chatbot only used basic information to complete the introduction, and no critical writing was obtained. For the mathematical case study, we observed that under the current status of the ChatGPT, this platform has no ability or skills to solve complex equations. Thus, significant changes need to be conducted to improve the performance of ChatGPT for mathematical solutions.