How Students Prompt ChatGPT for Creative Problem-Solving: Process Mining of Hybrid Human-AI Regulation

Background: The development of generative AI systems like ChatGPT has provoked debates about their effective use in educational settings. Aims: The present study explores how university students prompt ChatGPT to solve complex non-routine problems, specifically examining which prompts are associated with higher or lower problem-solving performance.Sample: Seventy-seven university students (53 women; Mage = 22.4 years) participated in the study. Methods: To identify various prompt types employed by students, the study utilized qualitative analysis of interactions with ChatGPT 3.5 during the resolution of the creative problem-solving task. Participants’ performance was measured by the quality, elaboration, and originality of their ideas. Subsequently, two-step clustering was employed to identify groups of low- and high-performing students. Finally, process mining techniques (heuristics miner) were used to analyze the interactions of low- and high-performing students. Results: The findings suggest that including clear evaluation criteria when prompting ChatGPT to generate ideas (rs = .38), providing ChatGPT with an elaborated context for idea generation (rs = .47), and offering specific feedback (rs = .45), enhances the quality, elaboration, and originality of the solutions. Successful problem-solving involves iterative human-AI regulation, with high performers using an overall larger number of prompts (d = 0.82). High performers interacted with ChatGPT through dialogue, where they monitored and regulated the generation of ideas, while low performers used ChatGPT as an information resource. Conclusions: These results emphasize the importance of active and iterative engagement for creative problem-solving and suggest that educational practices should foster metacognitive monitoring and regulation to maximize the benefits of human-AI collaboration.