Pre-Service Teachers‘ Approaches in Solving Mathematics Tasks with ChatGPT – A Qualitative Analysis of the Current Status Quo

Abstract This study investigates pre-service teachers’ competence in using large language models in mathematical contexts and with that the ability to evaluate answers provided by the chatbot. This is of interest as due to its probabilistic nature the answers cannot be foreseen while bearing the risk of being erroneous although sounding plausible. Eleven pre-service teachers were asked to solve four different tasks with the help of ChatGPT. The chatlogs and information provided in an interview after working on the tasks are analyzed using qualitative content analysis. Results show that both correct and incorrect answers were produced for all tasks. The rate of pre-service teacher providing an incorrect answer is high when having been presented an incorrect answer generated by the large language model. Despite having access to ChatGPT as a tool many of the participants were not able to live up to their self-evaluated mathematical skill and had trouble solving more complex problems. Furthermore, prompting techniques are analyzed and found to have a great variety while seemingly being dependent on the task as a context. Therefore, the focus of future research should lie on the user’s competence as well as the context the model is used in.