Generative AI in Assessment: AI Detectors and Implications for Practice

Abstract Generative AI has garnered attention as a valuable tool in the field of education. However, educators have expressed concerns about the originality of texts submitted by students as assignments due to the potential use of generative AI in writing tasks. Texts in writing assignments can be categorized into three types: AI-generated, human-written, and mixed texts (a combination of AI-generated and human-written texts). The purpose of the current study is to address some concerns from educators on the detection of the texts submitted by students, i.e., the consistency in detection results and accuracy of the detection results. The subject of the current study was the texts submitted by students as assignments for a graduate course. Four detectors were used to analyze the texts. Our findings provided useful information for educators: 1) Within the same detector, the consistency of the detection results for three types of texts were all above 90%. 2) Among different detectors, the detection results of human-written texts exhibited the highest consistency, whereas mixed texts demonstrated the lowest consistency.3) For accuracy, AI-generated and human-written texts were higher than mixed texts. Implications for educational practice were discussed.