Analyzing Discourse Patterns in ChatGPT-Generated IELTS Writing Task 2 Essays: A Discourse Analysis Approach

Analyzing Discourse Patterns in ChatGPT-Generated IELTS Writing Task 2 Essays: A Discourse Analysis Approach | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Analyzing Discourse Patterns in ChatGPT-Generated IELTS Writing Task 2 Essays: A Discourse Analysis Approach Pourya Javahery This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5608928/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract This study examines the discourse features of ChatGPT-generated IELTS Writing Task 2 essays, with a focus on the specific characteristics associated with band descriptors for scores of 6 and 7.Utilizing discourse and content analyses approaches, the studyexamines coherence, cohesion, argument structure, and lexical resource to understand the discourse characteristics indicative of different proficiency levels. Findings reveal that band 6 essays exhibit basic coherence with abrupt transitions and limited use of cohesive devices which result in a more linear argument structure and simpler lexical choices. Conversely, band 7 essays demonstrate a clearer progression of ideas, enhanced cohesion through varied cohesive devices, and a more complex argument structure that effectively integrates counterarguments and depth of analysis. These discourse patterns underscore the potential for AI-generated texts to model proficiency levels in writing and serve as pedagogical tools to improve learner outcomes. By highlighting the discourse elements critical to achieving higher band scores, this study contributes valuable insights into AI’s role in supporting language learning and academic writing proficiency. AI-generated texts IELTS Writing Task 2 discourse analysis writing proficiency pedagogy Introduction The advent of artificial intelligence (AI) in language generation represents a profound shift in the educational landscape, challenging traditional notions of authorship, creativity, and the very nature of learning itself (Livberber and Ayvaz, 2023). As AI technologies like ChatGPT emerge as tools for language production, teachers and researchers are compelled to re-examine the implications of these innovations on language acquisition and writing pedagogy. This shift invites us to reflect on the role of AI not merely as a tool but as a co-creator in the educational process which fosters a dialogue about the intersection of human cognition and machine intelligence (Markauskaite et al., 2022). The International English Language Testing System (IELTS) serves as a crucial benchmark for assessing language proficiency, emphasizing essential skills such as coherence, cohesion, and argument structure (Nguyen and Van Anh Le, 2024). These elements are not only foundational to academic writing but also reflective of broader communicative competencies that underpin meaningful discourse in our globalized world. By investigating the discourse patterns of ChatGPT-generated IELTS Writing Task 2 essays, this study seeks to illuminate the ways in which AI-generated texts can both complement and challenge our existing frameworks for understanding language use and writing assessment. Moreover, this exploration raises critical inquiries into the nature of learning itself. Can AI serve as a catalyst for deeper engagement with language, prompting learners to navigate the complexities of argumentation and discourse in novel ways? Conversely, does reliance on AI-generated content risk undermining the very essence of writing as a deeply personal and human endeavor? As we investigate the intricacies of discourse analysis, this study aims to uncover the patterns that characterize AI-generated texts, ultimately contributing to a richer understanding of how AI can be harnessed to enhance writing pedagogy and inform assessment practices. In addressing these questions, this research not only seeks to contribute to the academic discourse surrounding AI in education but also aspires to provoke thoughtful reflection among educators and learners alike about the future of writing in an increasingly automated world. Literature Review Discourse Analysis in Writing Assessment Discourse analysis (DA) plays a pivotal role in writing assessment by providing a framework for understanding how text structure, linguistic features, and communicative intent converge to create meaning. As Connor and Mbaye (2002) argue, the integration of discourse features into writing assessment is essential but often overlooked in favor of more surface-level criteria. They highlight a significant gap between current practices in writing assessment and the advances made in text analysis, particularly in examining deeper textual elements like coherence, argument structure, and cohesion. These insights suggest that writing assessments must evolve to capture a deeper understanding of students' communicative abilities. In this regard, Wang and Xie (2022) contribute valuable insights into how discourse competence in EFL academic writing can be diagnostically assessed through textual analysis, composing strategies, and academic writing knowledge. Their study emphasizes the importance of global coherence, showing that this essential discourse feature often challenges EFL students due to factors like time management and genre knowledge. They propose a targeted, three-stage instructional model to help learners build these discourse skills, highlighting the potential for more refined assessment frameworks that support learners’ mastery of coherence and cohesion in writing. Further, Spence (2010) explored how teachers use analytical rubrics to assess the writing of English Language Learners (ELLs), revealing that educators often focus on the descriptors for lower-score bands. This practice, while addressing students' immediate errors, may fail to leverage broader linguistic insights that DA could provide. This disconnection between rubric-based assessments and the broader linguistic knowledge of learners suggests a need for more comprehensive assessment frameworks that take into account the discourse-level features of writing. Expanding on this focus, Liontou (2022) examines discourse competence in secondary EFL learners, analyzing how specific discourse and linguistic features influence writing scores. In their study, features such as lexical diversity, word frequency, and sentence complexity were identified as significant indicators of language competence. These findings underscore the value of capturing detailed discourse features in writing assessments, with a particular focus on how language proficiency can influence the sophistication of text structure and vocabulary use among younger learners. Another key contribution to DA's role in language assessment comes from Frost (2021), who applies DA to speaking assessments. His work underscores the potential of discourse analysis to align test tasks more closely with real-world language demands, which has implications for writing assessments as well. Similarly, Burstein et al. (2003) describe an automated discourse analysis system for student essays, which employs machine learning to model teacher behavior in identifying key discourse elements. This automated system reflects an emerging trend toward using AI in educational assessments, signaling a growing recognition of discourse elements as critical markers of writing quality. As Liontou (2022) indicates, features like lexical diversity and coherence are essential in distinguishing proficiency levels, suggesting that DA-based automated systems could integrate such discourse insights to enhance both the validity and reliability of assessments. Collectively, these studies highlight the value of discourse analysis not only in the development of assessment tools like rubrics but also in automated scoring systems, enhancing both the validity and reliability of writing assessment. AI-Generated Writing Tools in Language Learning The growing use of AI-generated writing tools in language learning contexts has sparked significant research interest. These tools, particularly in second language (L2) learning, promise in enhancing text production and supporting learners at various proficiency levels. Simonsen (2021) and Tseng and Warschauer (2023) emphasize that AI-generated writing can assist learners by providing models of coherent, grammatically accurate texts, which can be especially valuable for students developing their academic writing skills. However, Teng (2024a)highlights several challenges and contradictions in using AI for language learning. While these tools can improve grammatical accuracy and the flow of texts, they often fall short in producing content that is dense, contextually relevant, or sophisticated in argumentation. In addition, Guo and Li (2024) explore an innovative approach where EFL students leverage AI through self-designed retrieval-augmented generation (RAG) chatbots to aid in various aspects of the writing process. Their findings reveal that students actively used chatbots to support idea generation, produce writing outlines, and address grammatical errors. Notably, creating personalized tools seemed to foster students' motivation and positively impacted their writing goals and confidence. These findings suggest that self-made AI tools can enhance learner autonomy and motivation, providing tailored support that aligns with individual writing needs. To address these challenges, Tseng and Warschauer (2023) propose a five-part pedagogical framework designed to help learners partner effectively with AI-generated writing tools. The framework focuses on understanding the capabilities of these tools, accessing and prompting AI systems, corroborating information provided by AI, and incorporating these technologies meaningfully into the learning process. This structured approach aims to ensure that learners not only use AI to generate grammatically accurate content but also critically evaluate and refine the output for greater coherence and argumentation. Escalante et al. (2023) contribute further to this discussion by examining AI feedback’s effectiveness in academic writing contexts. Their study on ENL learners revealed comparable learning outcomes between groups receiving human tutor feedback and those receiving AI-generated feedback. Interestingly, students expressed near-equal preference for AI or human feedback, suggesting the potential for a blended feedback approach in educational contexts. This study highlights the need for balanced integration, as students can benefit from the strengths of both AI and human feedback, ultimately supporting an approach to AI-assisted learning. Despite these promising aspects, Marzuki et al. (2023)point out that while AI writing assistants can improve surface-level features such as grammatical accuracy and coherence, they often struggle with content depth and relevance. This observation highlights the need for teachers to teach students how to critically engage with AI-generated texts (Teng, 2024b) Preparing students for a world where AI literacy is increasingly valued, as Warschauer et al. (2023) suggest, requires not only the integration of AI into the classroom but also focused instruction on its limitations and optimal use. The relevance of AI writing technologies in language learning contexts today parallels the introduction of calculators in math education decades ago, as noted by Simonsen (2021) which reflects a broader educational shift toward AI literacy. Coherence, Cohesion, and Argument Structure in IELTS Writing Research on IELTS writing, particularly the Writing Task 2 section, underscores the importance of key discourse elements such as coherence, cohesion, and argument structure. These components are essential in determining the clarity and persuasiveness of a written argument, which directly impacts IELTS scores. Arzhadeeva and Kudinova (2020) suggest that incorporating debate activities in classroom instruction can significantly improve students’ writing skills, particularly in areas of task response and coherence/cohesion. This finding points to the importance of teaching strategies that not only address language accuracy but also enhance students' ability to construct well-organized, logical essays. However, challenges remain in how these discourse features are assessed highlight that examiners often find coherence and cohesion to be the most challenging criteria to evaluate, leading to potential issues with construct validity in scoring. This difficulty emphasizes the need for clearer guidelines and training in assessing these elements (Vincheh et al. (2024); Ahmad, 2018; Burstein et al., 2003). Moreover, the role of corrective feedback in improving discourse features has been demonstrated in studies such asNguyen and Van Le (2022), which found that using model essays as feedback tools led to significant improvements in students’ task response and coherence/cohesion scores. This research suggests that targeted feedback, particularly in the form of high-quality model texts, can be an effective tool for developing learners' discourse competence. An analysis of cohesive devices used in IELTS essays by Yao (2014) revealed that reference was the most frequently used device, followed by conjunction and lexical cohesion. Substitution, however, was rarely used, reflecting a gap in students' mastery of diverse cohesive strategies. Yao's study also identified common cohesion problems, such as the misuse and overuse of conjunctions and references, often exacerbated by grammatical errors or inappropriate task responses. These findings highlight the importance of focused instruction on coherence and cohesion in IELTS writing preparation, as mastering these elements is critical for success in the exam. Rationale for the Present Study This study aims to fill a significant gap in research by examining ChatGPT-generated IELTS Writing Task 2 essays, specifically analyzing the discourse features associated with band scores 6 and 7. While prior research has largely focused on human-produced writing, there is limited exploration into how AI-generated texts align with IELTS band descriptors. By analyzing key discourse features such as coherence, cohesion, argument structure, and lexical resource, this study will assess how AI-generated essays reflect the proficiency levels outlined in the IELTS writing criteria. More specifically, it will investigate whether these AI-generated texts exhibit the characteristics of a well-structured argument, effective use of cohesive devices, and appropriate lexical variety, which are essential for achieving higher band scores. The findings will provide insights into whether AI-generated essays can serve as practical models for students, particularly in terms of helping learners understand the types of discourse features they need to improve in their own writing. To guide this investigation, the study addresses the following research question: What discourse features distinguish ChatGPT-generated IELTS Writing Task 2 essays at band scores 6 and 7 in terms of coherence, cohesion, argument structure, and lexical resource? Methodology Data Collection To ensure a diverse and representative dataset, a range of essay prompts was selected based on themes commonly encountered in IELTS Writing Task 2. These prompts covered a broad spectrum of topics, including education, technology, and environmental issues. The selection process was mindful of ensuring that no prompts were directly copied from official IELTS materials. Instead, the prompts were designed to reflect the general nature and style of common academic essay themes found in IELTS-like contexts. Text Generation For each selected prompt, ChatGPT was used to generate two distinct essays. One essay was designed to reflect a score of 6, while the other targeted a score of 7. These score levels were determined based on the IELTS Writing Task 2 band descriptors, which had been explained to ChatGPT prior to essay generation. This ensured that the essays demonstrated key characteristics typical of these band levels, such as differences in coherence, cohesion, lexical resource, and grammatical range. A total of 30 essays were generated, consisting of 15 prompts with two essays per prompt. Each essay adhered to a length of 250-300 words and followed a formal academic writing style, as expected in an IELTS Writing Task 2 response. Consistent parameters, such as essay length and prompt style, were maintained to provide a reliable basis for analysis. For each selected prompt, ChatGPT was asked to produce two distinct essays: Band 6 Essays : ChatGPT generated essays that simulated responses typically falling within the Band 6 range. These essays were characterized by moderate coherence and basic cohesion, with limited argument development and frequent errors in grammar and vocabulary. The sentences were simpler, with occasional breakdowns in clarity and precision in the use of cohesive devices. Band 7 Essays : ChatGPT then generated essays simulating the Band 7 range, with more developed arguments, clearer coherence, and more varied and accurate use of vocabulary and grammar. These essays presented a stronger progression of ideas and employed more sophisticated cohesive devices and sentence structures, though occasional lapses in accuracy were present. Each essay was generated within the standard word count of 250-300 words and followed formal academic writing conventions. In total, 32 essays were produced—two essays for each of the 16 prompts, one for Band 6 and one for Band 7. This approach allowed for an analysis of discourse patterns across different proficiency levels, as defined by the IELTS band descriptors. Data Analysis Discourse Analysis Framework Discourse Analysis (DA) is uniquely suited for this study as it provides a robust framework for examining the structural and functional aspects of language beyond surface-level features. The central focus of DA is on how language is used in context to convey meaning, organize information, and construct coherent arguments (Van Dijk, 2008) which makes it a critical tool for analyzing the effectiveness of communication in written texts. In the case of ChatGPT-generated IELTS Writing Task 2 essays, DA allows for a systematic exploration of how the AI model constructs discourse, particularly in terms of coherence, cohesion, and argument structure, which are key components in academic writing and language assessments. The discourse analysis (DA) in this study will focus on examining the structure and function of language used in the ChatGPT-generated IELTS Writing Task 2 essays. DA provides a framework for understanding how language operates in context, how meanings are constructed, and how these meanings interact with social and linguistic conventions, especially in an academic setting like the IELTS exam. Contextualizing Language Use In DA, one of the primary steps is to explore the context in which language is produced. For the purpose of this study, the context is framed by the IELTS Writing Task 2 format, where the essays are expected to demonstrate a formal, academic style. In analyzing ChatGPT-generated essays, the context will be scrutinized to see how well the AI model adheres to the expectations of formal discourse. This will include assessing whether the language is appropriately academic and if it reflects the type of discourse common in IELTS tasks. Coherence and Structure The DA will focus on global coherence, which refers to the logical arrangement and overall flow of ideas in each essay. This involves examining how effectively the essays follow a structured format, typical in academic writing, such as: Introduction with a clear thesis statement. Body paragraphs supporting the main argument with examples and evidence. Conclusion summarizing the points and restating the main claim. This phase of the analysis will assess whether the AI-generated essays follow these common academic structures and how well the ideas progress from one point to another. Cohesion and Linguistic Features The analysis will then turn to local coherence and cohesion within the essays. This involves analyzing the way ChatGPT uses cohesive devices such as conjunctions, references, ellipses, and lexical chains to connect sentences and ideas smoothly. The aim is to evaluate the appropriateness and variety of these cohesive features: Are cohesive devices such as "therefore," "however," "furthermore," used effectively? How does ChatGPT employ reference (e.g., pronouns, demonstratives) to maintain clarity across sentences? Does the use of lexical cohesion (repetition, synonyms, etc.) enhance the connectedness of the text, or does it lead to redundancy? This stage of the analysis will involve identifying both successful and problematic uses of cohesive devices to understand the strengths and weaknesses of AI discourse. Argumentation and Persuasion Next, the focus will shift to the argumentative discourse within the essays. Since IELTS Writing Task 2 requires an argumentative essay, it is crucial to analyze how effectively ChatGPT constructs arguments, persuades, and engages with different viewpoints: Does the essay present a clear stance or opinion in response to the prompt? How are counterarguments handled, and is there a balance between presenting a viewpoint and addressing opposing perspectives? Is the argumentation consistent and logically built upon throughout the essay? The discourse analysis will look at how arguments are framed and supported in AI-generated writing, paying attention to the clarity and complexity of the reasoning. Discourse Markers and Flow A key part of DA involves analyzing the use of discourse markers—words or phrases that guide the reader through the text, such as "firstly," "on the other hand," "in conclusion," etc. These markers are essential for ensuring clarity and smooth transitions between points in academic writing: Does ChatGPT employ discourse markers effectively to enhance the flow of the essay? Are these markers used too frequently or inappropriately, disrupting the natural progression of ideas? The effectiveness of discourse markers will be assessed to determine how well ChatGPT-generated essays manage transitions between arguments and maintain a logical structure. Textual Organization and Paragraphing Another important focus of DA in this study is the overall textual organization and paragraphing. DA will examine: How paragraphs are structured and whether they each focus on a single idea or topic. Whether the ideas within paragraphs are elaborated sufficiently with supporting details or whether they remain underdeveloped. How well the AI adheres to conventional paragraphing in academic writing, where each paragraph must contribute meaningfully to the argument or thesis of the essay. Through this analysis, patterns will emerge in terms of how well-organized the AI-generated essays are, revealing strengths or weaknesses in ChatGPT's capacity to create well-paragraphed, coherent texts. Patterns of Repetition and Formulaic Language One potential limitation of AI-generated writing is the tendency to rely on formulaic language and repetitive structures. The DA will investigate the frequency and variety of such patterns: Are there noticeable repetitions in how ChatGPT constructs its essays (e.g., starting every body paragraph with the same phrase or using similar sentence structures throughout)? Does the AI rely on formulaic expressions that might detract from the authenticity or sophistication of the discourse? Content Analysis Content Analysis is an appropriate method for this study as it allows for a detailed examination of the surface features and underlying patterns in ChatGPT-generated IELTS Writing Task 2 essays. This method involves systematically coding the texts to identify specific linguistic elements, such as cohesive devices, lexical choices, and sentence structures, which are crucial for evaluating coherence and cohesion in written discourse (Hazrati, 2023). Content Analysis enables the study to quantify the frequency and distribution of these elements, providing insights into how ChatGPT constructs meaning and organizes information within the constraints of formal academic writing. By focusing on measurable aspects of language use, Content Analysis offers a clear, data-driven foundation for assessing the alignment of AI-generated texts with established writing criteria, such as those found in the IELTS band descriptors. In this study, Content Analysis will serve as the primary method for examining the linguistic and discoursal features of ChatGPT-generated IELTS Writing Task 2 essays. The process will involve several key steps to ensure a thorough and systematic analysis of the texts, allowing for a detailed investigation into the patterns of coherence, cohesion, and argument structure. 1. Data Familiarization The first step will involve familiarizing with the data by reading through the 30 essays generated by ChatGPT based on a diverse set of IELTS prompts. This initial stage will allow for a comprehensive understanding of the texts, identifying general patterns and themes that will later inform the coding process. 2. Coding Process Once familiar with the essays, a coding scheme will be developed based on both the research questions and the key linguistic features outlined in the IELTS writing assessment criteria (e.g., coherence, cohesion, task response). The coding scheme will focus on identifying specific discourse features such as: Cohesive devices (e.g., conjunctions, transitions, lexical cohesion). Argument structure (e.g., thesis statement, supporting arguments, counterarguments). Logical flow (e.g., progression of ideas, paragraphing). Repetition and redundancy (e.g., overuse of certain linguistic structures). These features will be categorized according to their frequency and effectiveness in creating cohesive and coherent texts. 3. Quantifying Discourse Features Once the coding is complete, the frequency of each coded discourse feature will be quantified. This involves counting the occurrences of cohesive devices, transitions between paragraphs, and other relevant elements across the 30 essays. For example, how often ChatGPT uses linking words like "however," "therefore," or "in addition," and whether these devices are appropriately placed to ensure logical flow. Similarly, the presence and effectiveness of argument structures, such as clear thesis statements and the support for claims, will be assessed. 4. Categorizing Patterns After quantifying the discourse features, the next step will be to categorize these patterns based on their quality and effectiveness. Essays will be grouped into categories reflecting different levels of coherence and cohesion, aligning with the IELTS band descriptors (5-6 and 6-7). This categorization will help in identifying common trends within the AI-generated texts, such as consistent strengths or weaknesses in constructing cohesive and coherent arguments. 5. Thematic Analysis of Content In addition to the quantitative analysis, the coded data will also be analyzed thematically to explore more qualitative aspects of the texts. For instance, beyond counting cohesive devices, their effectiveness in supporting argument structure and enhancing logical flow will be examined. This qualitative component will allow for a deeper understanding of how ChatGPT mimics human-like discourse structures and whether its generated essays align with the discourse expectations of IELTS examiners. 6. Comparison of Band 6 and Band 7 Essays A comparative analysis will be conducted between essays aimed at achieving a band score of 5-6 and those targeted at a band score of 6-7. This comparison will highlight how variations in discourse features—such as the use of more sophisticated cohesive devices or a clearer argument structure—impact the overall quality of the essays. For example, essays aimed at a higher band score may exhibit greater variety and precision in the use of linking words, or they may demonstrate more complex argument structures with better-supported claims. 7. Identifying Gaps and Patterns Finally, the analysis will seek to identify gaps or inconsistencies in ChatGPT’s discourse generation. Are there areas where the AI consistently falls short, such as overusing certain cohesive devices or failing to effectively link arguments? The identification of these gaps will be crucial for understanding the limitations of AI-generated texts in high-stakes academic contexts like IELTS. Findings The analysis of the ChatGPT-generated IELTS Writing Task 2 essays reveals distinct patterns in discourse features, content organization, and argumentation styles that align with the band descriptors for scores 6 and 7 Discourse Analysis Coherence In the essays analyzed, coherence emerges as a crucial factor distinguishing the band scores. Band 6 essays exhibit a more rudimentary logical flow, often characterized by abrupt transitions between ideas. For instance, in the essay discussing the motivations for learning a foreign language, the shift from the advantages of travel to the cognitive benefits appears somewhat disjointed. Phrases like “On one hand” and “On the other hand” serve as transitional cues but do not adequately bridge the gap between differing perspectives. According to Crossley et al. (2016), such surface-level cohesive devices are frequently used by lower-level L2 writers as a compensatory strategy to create a semblance of structure. However, these transitions often lack the deep coherence required for a logical flow, as they do not effectively connect ideas on a conceptual level. This reliance on basic cohesive markers can hinder reader comprehension, as transitions appear disjointed and ideas feel fragmented. Conversely, band 7 essays display a clearer and more logical progression of ideas. The essay on economic progress articulates its argument effectively, weaving together concepts of economic growth, social welfare, and environmental considerations. Phrases such as “Moreover” and “In my opinion” enhance the logical flow, guiding the reader through the narrative with clarity and intent. This progression aligns with findings from Crossley and McNamara (2016), who emphasize that more proficient writers tend to employ cohesive devices that support deeper textual cohesion rather than simply marking transitions. By using phrases that build on previous arguments or connect ideas with better markers, these writers create an integrated argument that facilitates reader understanding and strengthens overall essay quality. Additionally, Nesi and Gardner (2012) suggest that logical coherence in academic writing reflects a mastery of genre expectations, where clear structuring around key themes is essential for effectively conveying complex ideas. Cohesion Cohesion in the essays reflects the effective use of cohesive devices, which can significantly impact the overall readability and connectivity of ideas. Band 6 essays tend to utilize a limited range of cohesive devices, primarily relying on simple conjunctions (e.g., “and,” “but”) to link sentences. This approach, while functional, may create a monotonous reading experience. For example, in the essay discussing transportation infrastructure, the phrase “On the other hand” introduces an opposing viewpoint but lacks variety in connecting ideas. As Crossley et al. (2016) point out, lower-level essays often depend on a narrow range of cohesive devices, which can lead to repetitive patterns and reduced reader engagement. This limited use of connectors suggests a lack of flexibility in managing linguistic resources, as writers are less able to select cohesive devices that enhance the flow of complex ideas. This, in turn, may hinder the overall cohesion and readability of the essay, as the lack of variation in conjunctions restricts the writer’s ability to create strong connections between points. In contrast, band 7 essays demonstrate a more sophisticated use of cohesive devices, incorporating a wider variety of references and conjunctions. In the same transportation essay, the use of terms like “furthermore” and “in addition” serves to enrich the argument, making the text more engaging and fluid. This diversity in cohesive devices not only enhances readability but also emphasizes the interrelationship between concepts, illustrating a deeper understanding of the subject matter. Research by Crossley and McNamara (2016) supports this finding by showing that higher-rated essays typically display a wider range of cohesive devices that contribute to smoother and more varied transitions. This variety in cohesion not only enhances readability but also indicates a greater ability to structure ideas in ways that highlight their interrelatedness which is a feature associated with higher essay quality. Additionally, Hyland and Jiang (2016) emphasize that effective cohesion in academic writing contributes to an impression of control and sophistication, as it shows the writer's capacity to guide the reader through complex arguments with clarity and intent. Argument Structure The structure of arguments within the essays reflects the complexity and depth of thought that can lead to higher band scores. Band 6 essays often present arguments in a more linear fashion, with basic topic sentences and supporting details. For instance, in the essay on information sharing, the argument regarding the benefits of open information is introduced but lacks substantial examples or in-depth analysis. Lower-scoring essays tend to follow a straightforward, additive structure, which may be adequate for basic clarity but limits the depth of engagement with the topic (Crossley et al., 2016). Such linear structures often reflect an undeveloped argumentation style where ideas are presented sequentially rather than interactively, reducing opportunities for critical analysis or exploration of multiple perspectives. In contrast, band 7 essays exhibit a stronger argument structure. They incorporate counterarguments effectively, illustrating a critical engagement with the topic. In the economic progress essay, the acknowledgment of social and environmental impacts enriches the discussion and demonstrates a comprehensive understanding of the issue. This aligns with findings from Nesi and Gardner (2012), who note that higher-quality academic writing often includes balanced argumentation that addresses multiple facets of a topic, fostering critical thinking and depth. By integrating counterarguments, these essays not only present a more sophisticated structure but also invite the reader to explore the issue’s complexity, thereby increasing the rhetorical appeal and perceived validity of the argument. Hyland and Jiang (2016) similarly emphasize that balanced argument structures in academic writing create a more dialogic style, engaging the reader actively and reflecting a more mature grasp of academic discourse. Such depth not only enhances the argument but also invites the reader to consider the complexity of real-world implications, showcasing a higher level of critical thinking. Content Analysis The content analysis further supports the distinction between band 6 and band 7 essays in terms of cohesive devices, argument structure, and common errors. Types of Cohesive Devices A comparative count of cohesive devices reveals that band 6 essays average fewer than ten cohesive devices, predominantly simple conjunctions. This limited range of cohesive devices often results in a repetitive or simplistic structure because ideas are connected in a straightforward, additive manner. For example, in an essay discussing the pros and cons of social media, a band 6 response used only “and” or “but” between points and created sentences such as “Social media is popular, and it allows people to connect easily, but it can be addictive.” With reliance on basic terms like “and” or “but,” band 6 essays can create a somewhat monotonous rhythm that may hinder the reader’s engagement and comprehension, as transitions lack complexity. Such a constrained approach reflects a limited lexical repertoire, where the writer may struggle to articulate subtle shifts in argument or introduce contrasting viewpoints effectively. In contrast, band 7 essays utilize over twenty cohesive devices, with a significant proportion being more advanced, such as “nevertheless,” “consequently,” and “in summary.” The use of these varied cohesive devices enables band 7 essays to establish smoother, more sophisticated transitions that guide the reader through complex arguments and shifts in perspective. For instance, in the same social media essay, a band 7 response said, “Social media has become essential for global communication; nevertheless, it can foster unhealthy habits among users.” These advanced devices serve not only as linguistic markers but also as signals of rhetorical intent which helps emphasize cause-effect relationships, concessions, or summative points. This ability to select and deploy a diverse array of cohesive devices indicates a higher level of lexical control, which enhances the overall cohesion of the text and elevates the essay’s readability and flow. By using more sophisticated cohesive devices, band 7 essays create a layered and dynamic argument structure that encourages the reader to engage deeply with the content, resulting in a more impactful and polished piece of writing. Structure of Arguments The examination of argument structure shows that band 6 essays frequently present a single viewpoint with minimal elaboration. For instance, in an essay discussing the demographic structure, a band 6 response outlined the benefits of a youthful population by stating simply that it leads to a “more energetic workforce” or “potential for innovation” without further explanation or support. This limited elaboration often leaves arguments feeling superficial and unconvincing, as the writer does not explore the implications or possible challenges associated with a younger demographic. Band 7 essays, however, present multiple perspectives and delve into each with greater depth. In a similar demographic essay at a band 7 level, the essay discussed both the advantages, such as increased productivity, and potential drawbacks, such as resource strain on education and health systems. They may include examples from countries with youthful populations, enhancing their argument with real-world relevance. The transportation essay also illustrates this well, where a band 7 response might offer a balanced exploration of both high-speed rail benefits and the limitations of existing public transport. . Common Errors Common errors in band 6 essays often include vague phrasing, limited vocabulary, and insufficient elaboration on key points. In contrast, band 7 essays show fewer such errors, with more precise language and a more extensive range of vocabulary. For example, a band 6 essay may describe the advantages of a young population as “good for the economy,” whereas a band 7 essay might articulate that a youthful demographic can “stimulate innovation and drive economic dynamism,” demonstrating a more sophisticated command of language. Table 1 Discourse Features and Band Score Distinctions in ChatGPT Essays Aspect Band 6 Essays Band 7 Essays Coherence Basic logical flow with abrupt transitions; limited cohesion in ideas. Ideas feel disconnected and transitions are often abrupt, making it harder to follow. Clearer progression of ideas with logical transitions; cohesive flow throughout. Transitions and relationships between ideas are clear, making the argument easy to follow. Cohesion Limited range of cohesive devices (e.g., simple conjunctions like “and,” “but”); repetitive and basic connectors. Results in a monotonous reading experience. Varied cohesive devices (e.g., “furthermore,” “in addition”); enriched readability and fluidity in connections. Diverse devices enhance text flow and create a more engaging experience. Argument Structure Simple, linear argument structure with basic topic sentences; minimal engagement with counterarguments. Arguments are often one-dimensional, lacking depth and multiple perspectives. Complex argument structure, integrating counterarguments and multiple perspectives; supports claims with depth. Argument is multifaceted and enriched with examples and counterpoints, showcasing critical thinking. Lexical Resource Limited vocabulary, often with vague phrasing; struggles with precise expression (e.g., “good for the economy”). Vocabulary lacks specificity which makes ideas sound general or imprecise. Wider vocabulary with more specific phrasing; articulates complex ideas (e.g., “stimulate innovation and dynamism”). Vocabulary is precise which allows for more accurate expression of complex concepts. Common Errors More frequent, including vague language, simplistic vocabulary, and underdeveloped points. Errors often detract from the clarity and persuasiveness of the essay. Fewer errors; demonstrates precise language, appropriate vocabulary, and depth in elaboration. Language is accurate and well-elaborated, contributing to a polished, coherent essay. Discussion The findings of this study illuminate the differences between the discourse features, content organization, and argumentation styles of ChatGPT-generated IELTS Writing Task 2 essays, revealing how these elements correlate with band scores of 6 and 7. By investigating these characteristics, this study uncovers not only the intricacies of effective writing but also the broader implications for language education in the context of emerging AI technologies. Coherence and Cohesion: A Philosophical Perspective The critical role of coherence in writing aligns with the philosophical notion of clarity in communication. Coherence serves as the backbone of effective discourse; it shapes the reader's journey through the text. Band 6 essays often faltered in this regard, presenting a rudimentary logical flow that hindered understanding. This observation echoes the work of scholars likeCrossley et al. (2016), who argue that coherence is vital for producing high-quality writing. The findings suggest that learners producing band 6 essays might be grappling with a lack of engagement with their material, a point made by Schmid (2020), who emphasizes the importance of narrative coherence in fostering understanding. In contrast, band 7 essays exhibited a sophisticated use of coherence and cohesion, employing a range of cohesive devices that enriched the narrative. This aligns with Hyland & Jiang (2016)’s assertion that effective academic writing relies on the strategic use of cohesion to guide readers through complex arguments. The significant difference in the application of cohesive devices underscores the necessity for writing instruction to prioritize these aspects, promoting an awareness of how language constructs meaning (Zahra et al., 2021) Argument Structure: Engaging with Complexity The examination of argument structures highlights the importance of depth in writing, reflecting the philosophical underpinnings of critical thinking and engagement with complexity. Band 6 essays often presented superficial arguments, failing to explore topics beyond a surface level. This finding resonates with the work ofTarchi et al. (2023), who emphasize that proficient writers engage in a recursive process of planning, translating, and reviewing their arguments. The lack of in-depth analysis in band 6 essays suggests a disconnect between the writer and the subject matter, hindering their ability to engage critically with diverse perspectives. Conversely, band 7 essays showcased a more intricate argument structure, incorporating counterarguments and perspectives. This complexity reflects the findings of a growing body of literature emphasizing the necessity of critical engagement in writing(Mao & Lee, 2023). By presenting multiple viewpoints, band 7 essays demonstrated an understanding of the multifaceted nature of real-world issues, inviting readers to consider the implications of various arguments. This underscores the importance of fostering a critical mindset in learners, encouraging them to explore and articulate diverse perspectives in their writing. Lexical Resource: The Power of Language The analysis of lexical choices between band scores reveals the profound impact of vocabulary on writing quality. Band 6 essays, characterized by vague phrasing and limited vocabulary, underscore the philosophical significance of language as a tool for expression. The work of Halliday (1985) emphasizes that language serves not only as a medium of communication but also as a means of constructing reality. The limited lexical resources observed in band 6 essays suggest a constricted worldview, which ultimately restricts the writer’s ability to convey complex ideas effectively. In contrast, band 7 essays demonstrated a more sophisticated command of language, employing varied vocabulary that enriched their arguments. This finding aligns with the assertions of Cazden (1988), who posits that a robust lexical resource enables writers to articulate their thoughts with precision. The disparity in vocabulary usage between the two bands suggests that educational practices must prioritize vocabulary development to foster a richer understanding of language and its expressive potential. Significance and Contribution of the Study This study contributes to the growing body of literature on AI and writing education by providing empirical evidence on the discourse features of AI-generated texts and their correlation with established writing proficiency standards. By analyzing the distinct characteristics of essays produced by ChatGPT, this research not only enriches our understanding of writing proficiency but also highlights the potential of AI in shaping language learning practices. Moreover, the findings underscore the importance of refining writing instruction to address the complexities of academic discourse. By emphasizing coherence, cohesive devices, argument structure, and lexical resource, educators can better equip learners with the skills necessary to navigate the challenges of academic writing. The implications of this study extend beyond IELTS preparation, offering valuable insights into the nature of effective communication in an increasingly digital landscape. Through the lens of discourse analysis, this study’s findings reveal key differences in the discourse features that mark varying levels of writing proficiency. By focusing on coherence, cohesion, argument structure, and lexical resource, we observe how AI-generated texts aim to replicate human discourse but often miss elements requiring deeper contextual understanding and authentic expression. This suggests that while AI can simulate certain language patterns, it lacks the situational awareness that human discourse naturally conveys. Furthermore, the distinctions between Band 6 and Band 7 essays go beyond surface features and reflect a varying engagement with language conventions specific to academic and high-stakes testing environments like IELTS. This reinforces the idea that discourse is influenced by communicative purpose and audience awareness. Teachers can leverage discourse analysis to help learners identify and apply effective discourse strategies, aiding them in understanding what is valued in academic writing. This study contributes to ongoing research into how discourse analysis can inform the development and assessment of AI-generated language, offering insights into both the potential and the limits of AI as a tool in discourse education. Conclusion This study has provided a comprehensive analysis of the discourse features and writing proficiency levels of ChatGPT-generated IELTS Writing Task 2 essays, revealing distinct patterns that align with the band descriptors for scores 6 and 7. The findings highlight the importance of coherence, cohesive devices, argument structure, and lexical resource in determining writing quality. Band 6 essays exhibited limitations in these areas, often presenting disjointed ideas and simplistic vocabulary, whereas band 7 essays demonstrated a more sophisticated command of language, showcasing complex argumentation and effective cohesion. The implications of this research extend beyond IELTS preparation, suggesting that AI-generated texts can serve as valuable tools for teaching writing. By focusing on the identified discourse features, educators can develop targeted instructional strategies to improve writing proficiency among learners. The integration of AI in language education presents an opportunity for personalized learning experiences, allowing students to engage critically with their writing processes and receive real-time feedback. Future Studies While this study offers valuable insights, it is not without limitations. First, the analysis is based solely on AI-generated texts, which may not fully represent the writing capabilities of human learners. Future research could explore the comparative analysis of AI-generated essays and those produced by actual students, providing a more holistic understanding of writing proficiency. Second, the study focused exclusively on IELTS Writing Task 2 essays, limiting the generalizability of the findings to other writing contexts or genres. Future investigations could examine different writing tasks across various academic disciplines to further validate the identified discourse features and their implications for writing instruction. Moreover, the study did not account for the potential variability in AI output based on different prompts or contexts, which may influence the discourse features and writing quality. Future research could employ a wider range of prompts to assess how variations in input impact the characteristics of AI-generated texts. Finally, exploring the perceptions of educators and learners regarding the use of AI-generated texts in writing education would provide valuable insights into the practical applications and challenges of integrating AI into language instruction. Understanding how these stakeholders perceive AI's role in writing can guide future pedagogical approaches and enhance the effectiveness of writing instruction in an increasingly digital age. In conclusion, this study opens avenues for further exploration into the intersection of AI and writing education, emphasizing the need for ongoing research to refine pedagogical practices and equip learners with the skills necessary to navigate the complexities of academic discourse in the 21st century. Declarations Funding No funds, grants, or other support was received. Consent for Publication All participants involved in this study provided their consent for the publication of their data and findings. Data Availability Data cannot be shared openly to protect study participant privacy. However, it is available and will be shared upon request. Competing interests Not applicable Acknowledgements Not applicable Clinical trial number Not applicable Author Contribution The author, Pourya Javahery, solely conceptualized, researched, and wrote the entire article, including the analysis and interpretation of data, drafting, and revising the manuscript. References Ahmad, Z. (2019). Analyzing Argumentative Essay as an Academic Genre on Assessment Framework of IELTS and TOEFL. In: Hidri, S. (eds) English Language Teaching Research in the Middle East and North Africa. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-319-98533-6_13 Arzhadeeva, D., & Kudinova, N. (2020). Using Debate to develop Writing Skills for IELTS Writing Task 2 among STEM Students. Journal of Language and Education , 6 (4), 30–43. https://doi.org/10.17323/jle.2020.10424 Burstein, J., Marcu, D., & Knight, K. (2003). Finding the WRITE stuff: automatic identification of discourse structure in student essays. IEEE Intelligent Systems , 18 (1), 32–39. https://doi.org/10.1109/mis.2003.1179191 Connor, U., & Mbaye, A. (2002). 14. DISCOURSE APPROACHES TO WRITING ASSESSMENT. Annual Review of Applied Linguistics , 22 , 263–278. https://doi.org/10.1017/s0267190502000144 Crossley, S. A., Kyle, K., & McNamara, D. S. (2016). The development and use of cohesive devices in L2 writing and their relations to judgments of essay quality. Journal of Second Language Writing , 32 , 1–16. https://doi.org/10.1016/j.jslw.2016.01.003 Crossley, S. A., & McNamara, D. S. (2016). Say more and be more coherent: How text elaboration and cohesion can increase writing quality. Journal of Writing Research , 7 (3), 351–370. https://doi.org/10.17239/jowr-2016.07.03.02 Escalante, J., Pack, A., & Barrett, A. (2023). AI-generated feedback on writing: insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education , 20 (1). https://doi.org/10.1186/s41239-023-00425-2 Gasaymeh, A. M., Beirat, M. A., & Qbeita, A. a. A. (2024). University students’ insights of Generative Artificial Intelligence (AI) writing tools. Education Sciences , 14 (10), 1062. https://doi.org/10.3390/educsci14101062 Guo, K., & Li, D. (2024). Understanding EFL students’ use of self-made AI chatbots as personalized writing assistance tools: A mixed methods study. System , 124 , 103362. https://doi.org/10.1016/j.system.2024.103362 Hazrati, Y. (2023). Content analysis of lexical cohesive devices in the lectures of the Minister of Health, Treatment and Medical Education of the Raisi Government in the field of health care, Treatment and health. International Journal of Multicultural and Multireligious Understanding , 10 (12), 439. https://doi.org/10.18415/ijmmu.v10i12.5446 Hyland, K., & Jiang, F. (2016). Is academic writing becoming more informal? English for Specific Purposes , 45 , 40–51. https://doi.org/10.1016/j.esp.2016.09.001 Liontou, T. (2022). Automated Discourse Analysis Techniques and Implications for Writing Assessment. Languages , 8 (1), 3. https://doi.org/10.3390/languages8010003 Livberber, T., & Ayvaz, S. (2023). The impact of Artificial Intelligence in academia: Views of Turkish academics on ChatGPT. Heliyon , 9 (9), e19688. https://doi.org/10.1016/j.heliyon.2023.e19688 Markauskaite, L., Marrone, R., Poquet, O., Knight, S., Martinez-Maldonado, R., Howard, S., Tondeur, J., De Laat, M., Shum, S. B., Gašević, D., & Siemens, G. (2022). Rethinking the entwinement between artificial intelligence and human learning: What capabilities do learners need for a world with AI? Computers and Education Artificial Intelligence , 3 , 100056. https://doi.org/10.1016/j.caeai.2022.100056 Marzuki, Widiati, U., Rusdin, D., Darwin, & Indrawati, I. (2023). The impact of AI writing tools on the content and organization of students’ writing: EFL teachers’ perspective. Cogent Education , 10 (2). https://doi.org/10.1080/2331186x.2023.2236469 Nesi, H., & Gardner, S. (2012). Genres across the disciplines: Student Writing in Higher Education . Cambridge University Press. Nguyen, H. T. M., & Van Anh Le, N. (2024). Text Complexity of Cambridge-delivered IELTS Academic Reading Tests: Comparability with IELTS Academic Reading Practice Tests from Other Publishers. Teaching English as a Second or Foreign Language--TESL-EJ , 28 (2). https://doi.org/10.55593/ej.28110a4 Nguyen, L.Q., Le, H.V. Improving L2 learners’ IELTS task 2 writing: the role of model essays and noticing hypothesis. Lang Test Asia 12 , 58 (2022). https://doi.org/10.1186/s40468-022-00206-0 Schmid, H. (2020). The dynamics of the linguistic system: Usage, Conventionalization, and Entrenchment . Oxford University Press. Spence, L. K. (2010). Discerning Writing Assessment: Insights into an Analytical Rubric. Language Arts , 87 (5), 337–352. https://doi.org/10.58680/la201010535 Teng, M. F. (2024a). A Systematic Review of ChatGPT for English as a Foreign Language Writing: Opportunities, Challenges, and Recommendations. International Journal of TESOL Studies , 6 (3). Teng, M. F. (2024b). “ChatGPT is the companion, not enemies”: EFL learners’ perceptions and experiences in using ChatGPT for feedback in writing. Computers and Education: Artificial Intelligence , 7 , 100270. Tseng, W., & Warschauer, M. (2023). AI-writing tools in education: if you can’t beat them, join them. Journal of China Computer-Assisted Language Learning , 3 (2), 258–262. https://doi.org/10.1515/jccall-2023-0008 Van Dijk, T. A. (2010). Discourse and context: A Sociocognitive Approach . Cambridge University Press. Vincheh, M. H., Mirzaei, A., & Roohani, A. (2024). A cognitive diagnostic approach to IELTS speaking test: unveiling the subskills and test-takers’ perceptions. Language Testing in Asia , 14(1). https://doi.org/10.1186/s40468-024-00311-2 Wang, Y., & Xie, Q. (2022). Diagnostic assessment of novice EFL learners’ discourse competence in academic writing: a case study. Language Testing in Asia , 12 (1). https://doi.org/10.1186/s40468-022-00197-y Warschauer, M., Tseng, W., Yim, S., Webster, T., Jacob, S., Du, Q., & Tate, T. (2023b). The affordances and contradictions of AI-generated text for writers of English as a second or foreign language. Journal of Second Language Writing , 62 , 101071. https://doi.org/10.1016/j.jslw.2023.101071 Yao, S. (2014). An analysis of Chinese students’ performance in IELTS academic writing. The New English Teacher , 8 (2). http://www.assumptionjournal.au.edu/index.php/newEnglishTeacher/article/download/295/253 Zahra, G. M., Emilia, E., & Nurlaelawati, I. (2021). An analysis of cohesion and coherence of descriptive texts written by junior high school students. Advances in Social Science, Education and Humanities Research/Advances in Social Science, Education and Humanities Research . https://doi.org/10.2991/assehr.k.210427.030 Additional Declarations No competing interests reported. Supplementary Files AppendixA.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5608928","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":395824108,"identity":"c2bda313-a9a5-409a-a115-42dc4e0eb1a5","order_by":0,"name":"Pourya Javahery","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABBElEQVRIiWNgGAWjYPACNiDmATFsgJix8QApWtJAWhqI0cIA03IYzMSrxZy9x+wzTw2fvMHxswcfV7adt1vbfhhoS41NNC4tlj1njGfzHGMz3HAmL9nwbNvt5G1nEoFajqXlNuDQYnAjx5iZh42NcWZDjplkI1CL2QGgFsaGwwS0/GOzn9n/xvxnY9u5ZLPzD4nQwtvGltgvkWPG2Nh2wM7sBgFbLHuOFTPO7WNL7pd4YyzZcC45wewG0JYEPH4xZ2/ezPDm2zHbNv4cw48NZXb2ZufTHz74UGOD22FAzMTDcAzCY2RjSASrTMChHKaF8QdDDZT7h8Eej+JRMApGwSgYoQAABapiPURkLwIAAAAASUVORK5CYII=","orcid":"","institution":"Islamic Azad University Central Tehran Branch","correspondingAuthor":true,"prefix":"","firstName":"Pourya","middleName":"","lastName":"Javahery","suffix":""}],"badges":[],"createdAt":"2024-12-09 12:23:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5608928/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5608928/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":72742609,"identity":"1c367f85-dbb0-491e-9710-c03be1c083ac","added_by":"auto","created_at":"2025-01-01 10:31:32","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":586270,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5608928/v1/47bd9b62-7c13-4510-a059-98a0aff47d92.pdf"},{"id":72729116,"identity":"7d83f68e-184c-44f2-a97b-08c90cc0cba2","added_by":"auto","created_at":"2025-01-01 05:34:01","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":292296,"visible":true,"origin":"","legend":"","description":"","filename":"AppendixA.docx","url":"https://assets-eu.researchsquare.com/files/rs-5608928/v1/00e605dedda4e9c49a818a1a.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Analyzing Discourse Patterns in ChatGPT-Generated IELTS Writing Task 2 Essays: A Discourse Analysis Approach","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe advent of artificial intelligence (AI) in language generation represents a profound shift in the educational landscape, challenging traditional notions of authorship, creativity, and the very nature of learning itself (Livberber and Ayvaz, 2023). \u0026nbsp;As AI technologies like ChatGPT emerge as tools for language production, teachers and researchers are compelled to re-examine the implications of these innovations on language acquisition and writing pedagogy. This shift invites us to reflect on the role of AI not merely as a tool but as a co-creator in the educational process which fosters a dialogue about the intersection of human cognition and machine intelligence (Markauskaite et al., 2022).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe International English Language Testing System (IELTS) serves as a crucial benchmark for assessing language proficiency, emphasizing essential skills such as coherence, cohesion, and argument structure (Nguyen and Van Anh Le, 2024). These elements are not only foundational to academic writing but also reflective of broader communicative competencies that underpin meaningful discourse in our globalized world. By investigating the discourse patterns of ChatGPT-generated IELTS Writing Task 2 essays, this study seeks to illuminate the ways in which AI-generated texts can both complement and challenge our existing frameworks for understanding language use and writing assessment.\u003c/p\u003e\n\u003cp\u003eMoreover, this exploration raises critical inquiries into the nature of learning itself. Can AI serve as a catalyst for deeper engagement with language, prompting learners to navigate the complexities of argumentation and discourse in novel ways? Conversely, does reliance on AI-generated content risk undermining the very essence of writing as a deeply personal and human endeavor? As we investigate the intricacies of discourse analysis, this study aims to uncover the patterns that characterize AI-generated texts, ultimately contributing to a richer understanding of how AI can be harnessed to enhance writing pedagogy and inform assessment practices.\u003c/p\u003e\n\u003cp\u003eIn addressing these questions, this research not only seeks to contribute to the academic discourse surrounding AI in education but also aspires to provoke thoughtful reflection among educators and learners alike about the future of writing in an increasingly automated world.\u003c/p\u003e"},{"header":"Literature Review","content":"\u003cp\u003e\u003cstrong\u003eDiscourse Analysis in Writing Assessment\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDiscourse analysis (DA) plays a pivotal role in writing assessment by providing a framework for understanding how text structure, linguistic features, and communicative intent converge to create meaning. As Connor and Mbaye (2002) argue, the integration of discourse features into writing assessment is essential but often overlooked in favor of more surface-level criteria. They highlight a significant gap between current practices in writing assessment and the advances made in text analysis, particularly in examining deeper textual elements like coherence, argument structure, and cohesion. These insights suggest that writing assessments must evolve to capture a deeper understanding of students\u0026apos; communicative abilities. In this regard, Wang and Xie (2022) contribute valuable insights into how discourse competence in EFL academic writing can be diagnostically assessed through textual analysis, composing strategies, and academic writing knowledge. Their study emphasizes the importance of global coherence, showing that this essential discourse feature often challenges EFL students due to factors like time management and genre knowledge. They propose a targeted, three-stage instructional model to help learners build these discourse skills, highlighting the potential for more refined assessment frameworks that support learners\u0026rsquo; mastery of coherence and cohesion in writing.\u003c/p\u003e\n\u003cp\u003eFurther, Spence (2010) explored how teachers use analytical rubrics to assess the writing of English Language Learners (ELLs), revealing that educators often focus on the descriptors for lower-score bands. This practice, while addressing students\u0026apos; immediate errors, may fail to leverage broader linguistic insights that DA could provide. This disconnection between rubric-based assessments and the broader linguistic knowledge of learners suggests a need for more comprehensive assessment frameworks that take into account the discourse-level features of writing. Expanding on this focus, Liontou (2022) examines discourse competence in secondary EFL learners, analyzing how specific discourse and linguistic features influence writing scores. In their study, features such as lexical diversity, word frequency, and sentence complexity were identified as significant indicators of language competence. These findings underscore the value of capturing detailed discourse features in writing assessments, with a particular focus on how language proficiency can influence the sophistication of text structure and vocabulary use among younger learners.\u003c/p\u003e\n\u003cp\u003eAnother key contribution to DA\u0026apos;s role in language assessment comes from Frost (2021), who applies DA to speaking assessments. His work underscores the potential of discourse analysis to align test tasks more closely with real-world language demands, which has implications for writing assessments as well. Similarly, Burstein et al. (2003) describe an automated discourse analysis system for student essays, which employs machine learning to model teacher behavior in identifying key discourse elements. This automated system reflects an emerging trend toward using AI in educational assessments, signaling a growing recognition of discourse elements as critical markers of writing quality. As Liontou (2022) indicates, features like lexical diversity and coherence are essential in distinguishing proficiency levels, suggesting that DA-based automated systems could integrate such discourse insights to enhance both the validity and reliability of assessments. Collectively, these studies highlight the value of discourse analysis not only in the development of assessment tools like rubrics but also in automated scoring systems, enhancing both the validity and reliability of writing assessment.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAI-Generated Writing Tools in Language Learning\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe growing use of AI-generated writing tools in language learning contexts has sparked significant research interest. These tools, particularly in second language (L2) learning, promise in enhancing text production and supporting learners at various proficiency levels. Simonsen (2021) and Tseng and Warschauer (2023) emphasize that AI-generated writing can assist learners by providing models of coherent, grammatically accurate texts, which can be especially valuable for students developing their academic writing skills. However, Teng (2024a)highlights several challenges and contradictions in using AI for language learning. While these tools can improve grammatical accuracy and the flow of texts, they often fall short in producing content that is dense, contextually relevant, or sophisticated in argumentation.\u003c/p\u003e\n\u003cp\u003eIn addition, Guo and Li (2024) explore an innovative approach where EFL students leverage AI through self-designed retrieval-augmented generation (RAG) chatbots to aid in various aspects of the writing process. Their findings reveal that students actively used chatbots to support idea generation, produce writing outlines, and address grammatical errors. Notably, creating personalized tools seemed to foster students\u0026apos; motivation and positively impacted their writing goals and confidence. These findings suggest that self-made AI tools can enhance learner autonomy and motivation, providing tailored support that aligns with individual writing needs.\u003c/p\u003e\n\u003cp\u003eTo address these challenges, Tseng and Warschauer (2023) propose a five-part pedagogical framework designed to help learners partner effectively with AI-generated writing tools. The framework focuses on understanding the capabilities of these tools, accessing and prompting AI systems, corroborating information provided by AI, and incorporating these technologies meaningfully into the learning process. This structured approach aims to ensure that learners not only use AI to generate grammatically accurate content but also critically evaluate and refine the output for greater coherence and argumentation. Escalante et al. (2023) contribute further to this discussion by examining AI feedback\u0026rsquo;s effectiveness in academic writing contexts. Their study on ENL learners revealed comparable learning outcomes between groups receiving human tutor feedback and those receiving AI-generated feedback. Interestingly, students expressed near-equal preference for AI or human feedback, suggesting the potential for a blended feedback approach in educational contexts. This study highlights the need for balanced integration, as students can benefit from the strengths of both AI and human feedback, ultimately supporting an approach to AI-assisted learning.\u003c/p\u003e\n\u003cp\u003eDespite these promising aspects, Marzuki et al. (2023)point out that while AI writing assistants can improve surface-level features such as grammatical accuracy and coherence, they often struggle with content depth and relevance. This observation highlights the need for teachers to teach students how to critically engage with AI-generated texts (Teng, 2024b) Preparing students for a world where AI literacy is increasingly valued, as Warschauer et al. (2023) suggest, requires not only the integration of AI into the classroom but also focused instruction on its limitations and optimal use. The relevance of AI writing technologies in language learning contexts today parallels the introduction of calculators in math education decades ago, as noted by Simonsen (2021) which reflects a broader educational shift toward AI literacy.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCoherence, Cohesion, and Argument Structure in IELTS Writing\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eResearch on IELTS writing, particularly the Writing Task 2 section, underscores the importance of key discourse elements such as coherence, cohesion, and argument structure. These components are essential in determining the clarity and persuasiveness of a written argument, which directly impacts IELTS scores. Arzhadeeva and Kudinova (2020) suggest that incorporating debate activities in classroom instruction can significantly improve students\u0026rsquo; writing skills, particularly in areas of task response and coherence/cohesion. This finding points to the importance of teaching strategies that not only address language accuracy but also enhance students\u0026apos; ability to construct well-organized, logical essays.\u003c/p\u003e\n\u003cp\u003eHowever, challenges remain in how these discourse features are assessed highlight that examiners often find coherence and cohesion to be the most challenging criteria to evaluate, leading to potential issues with construct validity in scoring. This difficulty emphasizes the need for clearer guidelines and training in assessing these elements (Vincheh et al. (2024); Ahmad, 2018; Burstein et al., 2003). Moreover, the role of corrective feedback in improving discourse features has been demonstrated in studies such asNguyen and Van Le (2022), which found that using model essays as feedback tools led to significant improvements in students\u0026rsquo; task response and coherence/cohesion scores. This research suggests that targeted feedback, particularly in the form of high-quality model texts, can be an effective tool for developing learners\u0026apos; discourse competence. An analysis of cohesive devices used in IELTS essays by Yao (2014) revealed that reference was the most frequently used device, followed by conjunction and lexical cohesion. Substitution, however, was rarely used, reflecting a gap in students\u0026apos; mastery of diverse cohesive strategies. Yao\u0026apos;s study also identified common cohesion problems, such as the misuse and overuse of conjunctions and references, often exacerbated by grammatical errors or inappropriate task responses. These findings highlight the importance of focused instruction on coherence and cohesion in IELTS writing preparation, as mastering these elements is critical for success in the exam.\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eRationale for the Present Study\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study aims to fill a significant gap in research by examining ChatGPT-generated IELTS Writing Task 2 essays, specifically analyzing the discourse features associated with band scores 6 and 7. While prior research has largely focused on human-produced writing, there is limited exploration into how AI-generated texts align with IELTS band descriptors. By analyzing key discourse features such as coherence, cohesion, argument structure, and lexical resource, this study will assess how AI-generated essays reflect the proficiency levels outlined in the IELTS writing criteria. More specifically, it will investigate whether these AI-generated texts exhibit the characteristics of a well-structured argument, effective use of cohesive devices, and appropriate lexical variety, which are essential for achieving higher band scores. The findings will provide insights into whether AI-generated essays can serve as practical models for students, particularly in terms of helping learners understand the types of discourse features they need to improve in their own writing.\u003c/p\u003e\n\u003cp\u003eTo guide this investigation, the study addresses the following research question:\u003c/p\u003e\n\u003cp\u003eWhat discourse features distinguish ChatGPT-generated IELTS Writing Task 2 essays at band scores 6 and 7 in terms of coherence, cohesion, argument structure, and lexical resource?\u003c/p\u003e"},{"header":"Methodology","content":"\u003cp\u003e\u003cstrong\u003eData Collection\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo ensure a diverse and representative dataset, a range of essay prompts was selected based on themes commonly encountered in IELTS Writing Task 2. These prompts covered a broad spectrum of topics, including education, technology, and environmental issues. The selection process was mindful of ensuring that no prompts were directly copied from official IELTS materials. Instead, the prompts were designed to reflect the general nature and style of common academic essay themes found in IELTS-like contexts.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eText Generation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFor each selected prompt, ChatGPT was used to generate two distinct essays. One essay was designed to reflect a score of 6, while the other targeted a score of 7. These score levels were determined based on the IELTS Writing Task 2 band descriptors, which had been explained to ChatGPT prior to essay generation. This ensured that the essays demonstrated key characteristics typical of these band levels, such as differences in coherence, cohesion, lexical resource, and grammatical range.\u003c/p\u003e\n\u003cp\u003eA total of 30 essays were generated, consisting of 15 prompts with two essays per prompt. Each essay adhered to a length of 250-300 words and followed a formal academic writing style, as expected in an IELTS Writing Task 2 response. Consistent parameters, such as essay length and prompt style, were maintained to provide a reliable basis for analysis.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFor each selected prompt, ChatGPT was asked to produce two distinct essays:\u003c/p\u003e\n\u003col start=\"1\" type=\"1\"\u003e\n \u003cli\u003e\u003cstrong\u003eBand 6 Essays\u003c/strong\u003e:\u003cul type=\"circle\"\u003e\n \u003cli\u003eChatGPT generated essays that simulated responses typically falling within the Band 6 range. These essays were characterized by moderate coherence and basic cohesion, with limited argument development and frequent errors in grammar and vocabulary. The sentences were simpler, with occasional breakdowns in clarity and precision in the use of cohesive devices.\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eBand 7 Essays\u003c/strong\u003e:\u003cul type=\"circle\"\u003e\n \u003cli\u003eChatGPT then generated essays simulating the Band 7 range, with more developed arguments, clearer coherence, and more varied and accurate use of vocabulary and grammar. These essays presented a stronger progression of ideas and employed more sophisticated cohesive devices and sentence structures, though occasional lapses in accuracy were present.\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eEach essay was generated within the standard word count of 250-300 words and followed formal academic writing conventions. In total, 32 essays were produced\u0026mdash;two essays for each of the 16 prompts, one for Band 6 and one for Band 7. This approach allowed for an analysis of discourse patterns across different proficiency levels, as defined by the IELTS band descriptors.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDiscourse Analysis Framework\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDiscourse Analysis (DA) is uniquely suited for this study as it provides a robust framework for examining the structural and functional aspects of language beyond surface-level features. The central focus of DA is on how language is used in context to convey meaning, organize information, and construct coherent arguments (Van Dijk, 2008) which makes it a critical tool for analyzing the effectiveness of communication in written texts. In the case of ChatGPT-generated IELTS Writing Task 2 essays, DA allows for a systematic exploration of how the AI model constructs discourse, particularly in terms of coherence, cohesion, and argument structure, which are key components in academic writing and language assessments.\u003c/p\u003e\n\u003cp\u003eThe discourse analysis (DA) in this study will focus on examining the structure and function of language used in the ChatGPT-generated IELTS Writing Task 2 essays. DA provides a framework for understanding how language operates in context, how meanings are constructed, and how these meanings interact with social and linguistic conventions, especially in an academic setting like the IELTS exam.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eContextualizing Language Use\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn DA, one of the primary steps is to explore the context in which language is produced. For the purpose of this study, the context is framed by the IELTS Writing Task 2 format, where the essays are expected to demonstrate a formal, academic style. In analyzing ChatGPT-generated essays, the context will be scrutinized to see how well the AI model adheres to the expectations of formal discourse. This will include assessing whether the language is appropriately academic and if it reflects the type of discourse common in IELTS tasks.\u003cstrong\u003eCoherence and Structure\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe DA will focus on global coherence, which refers to the logical arrangement and overall flow of ideas in each essay. This involves examining how effectively the essays follow a structured format, typical in academic writing, such as:\u003c/p\u003e\n\u003cul type=\"disc\"\u003e\n \u003cli\u003eIntroduction with a clear thesis statement.\u003c/li\u003e\n \u003cli\u003eBody paragraphs supporting the main argument with examples and evidence.\u003c/li\u003e\n \u003cli\u003eConclusion summarizing the points and restating the main claim.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThis phase of the analysis will assess whether the AI-generated essays follow these common academic structures and how well the ideas progress from one point to another.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCohesion and Linguistic Features\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe analysis will then turn to local coherence and cohesion within the essays. This involves analyzing the way ChatGPT uses cohesive devices such as conjunctions, references, ellipses, and lexical chains to connect sentences and ideas smoothly. The aim is to evaluate the appropriateness and variety of these cohesive features:\u003c/p\u003e\n\u003cul type=\"disc\"\u003e\n \u003cli\u003eAre cohesive devices such as \u0026quot;therefore,\u0026quot; \u0026quot;however,\u0026quot; \u0026quot;furthermore,\u0026quot; used effectively?\u003c/li\u003e\n \u003cli\u003eHow does ChatGPT employ reference (e.g., pronouns, demonstratives) to maintain clarity across sentences?\u003c/li\u003e\n \u003cli\u003eDoes the use of lexical cohesion (repetition, synonyms, etc.) enhance the connectedness of the text, or does it lead to redundancy?\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThis stage of the analysis will involve identifying both successful and problematic uses of cohesive devices to understand the strengths and weaknesses of AI discourse.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eArgumentation and Persuasion\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNext, the focus will shift to the argumentative discourse within the essays. Since IELTS Writing Task 2 requires an argumentative essay, it is crucial to analyze how effectively ChatGPT constructs arguments, persuades, and engages with different viewpoints:\u003c/p\u003e\n\u003cul type=\"disc\"\u003e\n \u003cli\u003eDoes the essay present a clear stance or opinion in response to the prompt?\u003c/li\u003e\n \u003cli\u003eHow are counterarguments handled, and is there a balance between presenting a viewpoint and addressing opposing perspectives?\u003c/li\u003e\n \u003cli\u003eIs the argumentation consistent and logically built upon throughout the essay?\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe discourse analysis will look at how arguments are framed and supported in AI-generated writing, paying attention to the clarity and complexity of the reasoning.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDiscourse Markers and Flow\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA key part of DA involves analyzing the use of discourse markers\u0026mdash;words or phrases that guide the reader through the text, such as \u0026quot;firstly,\u0026quot; \u0026quot;on the other hand,\u0026quot; \u0026quot;in conclusion,\u0026quot; etc. These markers are essential for ensuring clarity and smooth transitions between points in academic writing:\u003c/p\u003e\n\u003cul type=\"disc\"\u003e\n \u003cli\u003eDoes ChatGPT employ discourse markers effectively to enhance the flow of the essay?\u003c/li\u003e\n \u003cli\u003eAre these markers used too frequently or inappropriately, disrupting the natural progression of ideas?\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe effectiveness of discourse markers will be assessed to determine how well ChatGPT-generated essays manage transitions between arguments and maintain a logical structure.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTextual Organization and Paragraphing\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAnother important focus of DA in this study is the overall textual organization and paragraphing. DA will examine:\u003c/p\u003e\n\u003cul type=\"disc\"\u003e\n \u003cli\u003eHow paragraphs are structured and whether they each focus on a single idea or topic.\u003c/li\u003e\n \u003cli\u003eWhether the ideas within paragraphs are elaborated sufficiently with supporting details or whether they remain underdeveloped.\u003c/li\u003e\n \u003cli\u003eHow well the AI adheres to conventional paragraphing in academic writing, where each paragraph must contribute meaningfully to the argument or thesis of the essay.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThrough this analysis, patterns will emerge in terms of how well-organized the AI-generated essays are, revealing strengths or weaknesses in ChatGPT\u0026apos;s capacity to create well-paragraphed, coherent texts.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePatterns of Repetition and Formulaic Language\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eOne potential limitation of AI-generated writing is the tendency to rely on formulaic language and repetitive structures. The DA will investigate the frequency and variety of such patterns:\u003c/p\u003e\n\u003cul type=\"disc\"\u003e\n \u003cli\u003eAre there noticeable repetitions in how ChatGPT constructs its essays (e.g., starting every body paragraph with the same phrase or using similar sentence structures throughout)?\u003c/li\u003e\n \u003cli\u003eDoes the AI rely on formulaic expressions that might detract from the authenticity or sophistication of the discourse?\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003eContent Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eContent Analysis is an appropriate method for this study as it allows for a detailed examination of the surface features and underlying patterns in ChatGPT-generated IELTS Writing Task 2 essays. This method involves systematically coding the texts to identify specific linguistic elements, such as cohesive devices, lexical choices, and sentence structures, which are crucial for evaluating coherence and cohesion in written discourse (Hazrati, 2023). Content Analysis enables the study to quantify the frequency and distribution of these elements, providing insights into how ChatGPT constructs meaning and organizes information within the constraints of formal academic writing. By focusing on measurable aspects of language use, Content Analysis offers a clear, data-driven foundation for assessing the alignment of AI-generated texts with established writing criteria, such as those found in the IELTS band descriptors.\u003c/p\u003e\n\u003cp\u003eIn this study, Content Analysis will serve as the primary method for examining the linguistic and discoursal features of ChatGPT-generated IELTS Writing Task 2 essays. The process will involve several key steps to ensure a thorough and systematic analysis of the texts, allowing for a detailed investigation into the patterns of coherence, cohesion, and argument structure.\u003c/p\u003e\n\u003cp\u003e1. Data Familiarization\u003c/p\u003e\n\u003cp\u003eThe first step will involve familiarizing with the data by reading through the 30 essays generated by ChatGPT based on a diverse set of IELTS prompts. This initial stage will allow for a comprehensive understanding of the texts, identifying general patterns and themes that will later inform the coding process.\u003c/p\u003e\n\u003cp\u003e2. Coding Process\u003c/p\u003e\n\u003cp\u003eOnce familiar with the essays, a coding scheme will be developed based on both the research questions and the key linguistic features outlined in the IELTS writing assessment criteria (e.g., coherence, cohesion, task response). The coding scheme will focus on identifying specific discourse features such as:\u003c/p\u003e\n\u003cul type=\"disc\"\u003e\n \u003cli\u003eCohesive devices (e.g., conjunctions, transitions, lexical cohesion).\u003c/li\u003e\n \u003cli\u003eArgument structure (e.g., thesis statement, supporting arguments, counterarguments).\u003c/li\u003e\n \u003cli\u003eLogical flow (e.g., progression of ideas, paragraphing).\u003c/li\u003e\n \u003cli\u003eRepetition and redundancy (e.g., overuse of certain linguistic structures). These features will be categorized according to their frequency and effectiveness in creating cohesive and coherent texts.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e3. Quantifying Discourse Features\u003c/p\u003e\n\u003cp\u003eOnce the coding is complete, the frequency of each coded discourse feature will be quantified. This involves counting the occurrences of cohesive devices, transitions between paragraphs, and other relevant elements across the 30 essays. For example, how often ChatGPT uses linking words like \u0026quot;however,\u0026quot; \u0026quot;therefore,\u0026quot; or \u0026quot;in addition,\u0026quot; and whether these devices are appropriately placed to ensure logical flow. Similarly, the presence and effectiveness of argument structures, such as clear thesis statements and the support for claims, will be assessed.\u003c/p\u003e\n\u003cp\u003e4. Categorizing Patterns\u003c/p\u003e\n\u003cp\u003eAfter quantifying the discourse features, the next step will be to categorize these patterns based on their quality and effectiveness. Essays will be grouped into categories reflecting different levels of coherence and cohesion, aligning with the IELTS band descriptors (5-6 and 6-7). This categorization will help in identifying common trends within the AI-generated texts, such as consistent strengths or weaknesses in constructing cohesive and coherent arguments.\u003c/p\u003e\n\u003cp\u003e5. Thematic Analysis of Content\u003c/p\u003e\n\u003cp\u003eIn addition to the quantitative analysis, the coded data will also be analyzed thematically to explore more qualitative aspects of the texts. For instance, beyond counting cohesive devices, their effectiveness in supporting argument structure and enhancing logical flow will be examined. This qualitative component will allow for a deeper understanding of how ChatGPT mimics human-like discourse structures and whether its generated essays align with the discourse expectations of IELTS examiners.\u003c/p\u003e\n\u003cp\u003e6. Comparison of Band 6 and Band 7 Essays\u003c/p\u003e\n\u003cp\u003eA comparative analysis will be conducted between essays aimed at achieving a band score of 5-6 and those targeted at a band score of 6-7. This comparison will highlight how variations in discourse features\u0026mdash;such as the use of more sophisticated cohesive devices or a clearer argument structure\u0026mdash;impact the overall quality of the essays. For example, essays aimed at a higher band score may exhibit greater variety and precision in the use of linking words, or they may demonstrate more complex argument structures with better-supported claims.\u003c/p\u003e\n\u003cp\u003e7. Identifying Gaps and Patterns\u003c/p\u003e\n\u003cp\u003eFinally, the analysis will seek to identify gaps or inconsistencies in ChatGPT\u0026rsquo;s discourse generation. Are there areas where the AI consistently falls short, such as overusing certain cohesive devices or failing to effectively link arguments? The identification of these gaps will be crucial for understanding the limitations of AI-generated texts in high-stakes academic contexts like IELTS.\u003c/p\u003e"},{"header":"Findings","content":"\u003cp\u003eThe analysis of the ChatGPT-generated IELTS Writing Task 2 essays reveals distinct patterns in discourse features, content organization, and argumentation styles that align with the band descriptors for scores 6 and 7\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDiscourse Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eCoherence\u003c/p\u003e\n\u003cp\u003eIn the essays analyzed, coherence emerges as a crucial factor distinguishing the band scores. Band 6 essays exhibit a more rudimentary logical flow, often characterized by abrupt transitions between ideas. For instance, in the essay discussing the motivations for learning a foreign language, the shift from the advantages of travel to the cognitive benefits appears somewhat disjointed. Phrases like \u0026ldquo;On one hand\u0026rdquo; and \u0026ldquo;On the other hand\u0026rdquo; serve as transitional cues but do not adequately bridge the gap between differing perspectives. According to Crossley et al. (2016), such surface-level cohesive devices are frequently used by lower-level L2 writers as a compensatory strategy to create a semblance of structure. However, these transitions often lack the deep coherence required for a logical flow, as they do not effectively connect ideas on a conceptual level. This reliance on basic cohesive markers can hinder reader comprehension, as transitions appear disjointed and ideas feel fragmented.\u003c/p\u003e\n\u003cp\u003eConversely, band 7 essays display a clearer and more logical progression of ideas. The essay on economic progress articulates its argument effectively, weaving together concepts of economic growth, social welfare, and environmental considerations. Phrases such as \u0026ldquo;Moreover\u0026rdquo; and \u0026ldquo;In my opinion\u0026rdquo; enhance the logical flow, guiding the reader through the narrative with clarity and intent. This progression aligns with findings from Crossley and McNamara (2016), who emphasize that more proficient writers tend to employ cohesive devices that support deeper textual cohesion rather than simply marking transitions. By using phrases that build on previous arguments or connect ideas with better markers, these writers create an integrated argument that facilitates reader understanding and strengthens overall essay quality. Additionally, Nesi and Gardner (2012) suggest that logical coherence in academic writing reflects a mastery of genre expectations, where clear structuring around key themes is essential for effectively conveying complex ideas.\u003c/p\u003e\n\u003cp\u003eCohesion\u003c/p\u003e\n\u003cp\u003eCohesion in the essays reflects the effective use of cohesive devices, which can significantly impact the overall readability and connectivity of ideas. Band 6 essays tend to utilize a limited range of cohesive devices, primarily relying on simple conjunctions (e.g., \u0026ldquo;and,\u0026rdquo; \u0026ldquo;but\u0026rdquo;) to link sentences. This approach, while functional, may create a monotonous reading experience. For example, in the essay discussing transportation infrastructure, the phrase \u0026ldquo;On the other hand\u0026rdquo; introduces an opposing viewpoint but lacks variety in connecting ideas. As Crossley et al. (2016) point out, lower-level essays often depend on a narrow range of cohesive devices, which can lead to repetitive patterns and reduced reader engagement. This limited use of connectors suggests a lack of flexibility in managing linguistic resources, as writers are less able to select cohesive devices that enhance the flow of complex ideas. This, in turn, may hinder the overall cohesion and readability of the essay, as the lack of variation in conjunctions restricts the writer\u0026rsquo;s ability to create strong connections between points.\u003c/p\u003e\n\u003cp\u003eIn contrast, band 7 essays demonstrate a more sophisticated use of cohesive devices, incorporating a wider variety of references and conjunctions. In the same transportation essay, the use of terms like \u0026ldquo;furthermore\u0026rdquo; and \u0026ldquo;in addition\u0026rdquo; serves to enrich the argument, making the text more engaging and fluid. This diversity in cohesive devices not only enhances readability but also emphasizes the interrelationship between concepts, illustrating a deeper understanding of the subject matter. Research by Crossley and McNamara (2016) supports this finding by showing that higher-rated essays typically display a wider range of cohesive devices that contribute to smoother and more varied transitions. This variety in cohesion not only enhances readability but also indicates a greater ability to structure ideas in ways that highlight their interrelatedness which is a feature associated with higher essay quality. Additionally, Hyland and Jiang (2016) emphasize that effective cohesion in academic writing contributes to an impression of control and sophistication, as it shows the writer\u0026apos;s capacity to guide the reader through complex arguments with clarity and intent.\u003c/p\u003e\n\u003cp\u003eArgument Structure\u003c/p\u003e\n\u003cp\u003eThe structure of arguments within the essays reflects the complexity and depth of thought that can lead to higher band scores. Band 6 essays often present arguments in a more linear fashion, with basic topic sentences and supporting details. For instance, in the essay on information sharing, the argument regarding the benefits of open information is introduced but lacks substantial examples or in-depth analysis. Lower-scoring essays tend to follow a straightforward, additive structure, which may be adequate for basic clarity but limits the depth of engagement with the topic (Crossley et al., 2016). Such linear structures often reflect an undeveloped argumentation style where ideas are presented sequentially rather than interactively, reducing opportunities for critical analysis or exploration of multiple perspectives.\u003c/p\u003e\n\u003cp\u003eIn contrast, band 7 essays exhibit a stronger argument structure. They incorporate counterarguments effectively, illustrating a critical engagement with the topic. In the economic progress essay, the acknowledgment of social and environmental impacts enriches the discussion and demonstrates a comprehensive understanding of the issue. This aligns with findings from Nesi and Gardner (2012), who note that higher-quality academic writing often includes balanced argumentation that addresses multiple facets of a topic, fostering critical thinking and depth. By integrating counterarguments, these essays not only present a more sophisticated structure but also invite the reader to explore the issue\u0026rsquo;s complexity, thereby increasing the rhetorical appeal and perceived validity of the argument. Hyland and Jiang (2016) similarly emphasize that balanced argument structures in academic writing create a more dialogic style, engaging the reader actively and reflecting a more mature grasp of academic discourse. Such depth not only enhances the argument but also invites the reader to consider the complexity of real-world implications, showcasing a higher level of critical thinking.\u003c/p\u003e\n\u003cp\u003eContent Analysis\u003c/p\u003e\n\u003cp\u003eThe content analysis further supports the distinction between band 6 and band 7 essays in terms of cohesive devices, argument structure, and common errors.\u003c/p\u003e\n\u003cp\u003eTypes of Cohesive Devices\u003c/p\u003e\n\u003cp\u003eA comparative count of cohesive devices reveals that band 6 essays average fewer than ten cohesive devices, predominantly simple conjunctions. This limited range of cohesive devices often results in a repetitive or simplistic structure because ideas are connected in a straightforward, additive manner. For example, in an essay discussing the pros and cons of social media, a band 6 response used only \u0026ldquo;and\u0026rdquo; or \u0026ldquo;but\u0026rdquo; between points and created sentences such as \u0026ldquo;Social media is popular, and it allows people to connect easily, but it can be addictive.\u0026rdquo; With reliance on basic terms like \u0026ldquo;and\u0026rdquo; or \u0026ldquo;but,\u0026rdquo; band 6 essays can create a somewhat monotonous rhythm that may hinder the reader\u0026rsquo;s engagement and comprehension, as transitions lack complexity. Such a constrained approach reflects a limited lexical repertoire, where the writer may struggle to articulate subtle shifts in argument or introduce contrasting viewpoints effectively.\u003c/p\u003e\n\u003cp\u003eIn contrast, band 7 essays utilize over twenty cohesive devices, with a significant proportion being more advanced, such as \u0026ldquo;nevertheless,\u0026rdquo; \u0026ldquo;consequently,\u0026rdquo; and \u0026ldquo;in summary.\u0026rdquo; \u0026nbsp;The use of these varied cohesive devices enables band 7 essays to establish smoother, more sophisticated transitions that guide the reader through complex arguments and shifts in perspective. For instance, in the same social media essay, a band 7 response said, \u0026ldquo;Social media has become essential for global communication; nevertheless, it can foster unhealthy habits among users.\u0026rdquo; These advanced devices serve not only as linguistic markers but also as signals of rhetorical intent which helps emphasize cause-effect relationships, concessions, or summative points. This ability to select and deploy a diverse array of cohesive devices indicates a higher level of lexical control, which enhances the overall cohesion of the text and elevates the essay\u0026rsquo;s readability and flow. By using more sophisticated cohesive devices, band 7 essays create a layered and dynamic argument structure that encourages the reader to engage deeply with the content, resulting in a more impactful and polished piece of writing.\u003c/p\u003e\n\u003cp\u003eStructure of Arguments\u003c/p\u003e\n\u003cp\u003eThe examination of argument structure shows that band 6 essays frequently present a single viewpoint with minimal elaboration. For instance, in an essay discussing the demographic structure, a band 6 response outlined the benefits of a youthful population by stating simply that it leads to a \u0026ldquo;more energetic workforce\u0026rdquo; or \u0026ldquo;potential for innovation\u0026rdquo; without further explanation or support. This limited elaboration often leaves arguments feeling superficial and unconvincing, as the writer does not explore the implications or possible challenges associated with a younger demographic.\u003c/p\u003e\n\u003cp\u003eBand 7 essays, however, present multiple perspectives and delve into each with greater depth. In a similar demographic essay at a band 7 level, the essay discussed both the advantages, such as increased productivity, and potential drawbacks, such as resource strain on education and health systems. They may include examples from countries with youthful populations, enhancing their argument with real-world relevance. The transportation essay also illustrates this well, where a band 7 response might offer a balanced exploration of both high-speed rail benefits and the limitations of existing public transport. .\u003c/p\u003e\n\u003cp\u003eCommon Errors\u003c/p\u003e\n\u003cp\u003eCommon errors in band 6 essays often include vague phrasing, limited vocabulary, and insufficient elaboration on key points. In contrast, band 7 essays show fewer such errors, with more precise language and a more extensive range of vocabulary. For example, a band 6 essay may describe the advantages of a young population as \u0026ldquo;good for the economy,\u0026rdquo; whereas a band 7 essay might articulate that a youthful demographic can \u0026ldquo;stimulate innovation and drive economic dynamism,\u0026rdquo; demonstrating a more sophisticated command of language.\u003c/p\u003e\n\u003cp\u003eTable 1 Discourse Features and Band Score Distinctions in ChatGPT Essays\u003c/p\u003e\n\u003ctable border=\"0\" cellpadding=\"0\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eAspect\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eBand 6 Essays\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eBand 7 Essays\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eCoherence\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eBasic logical flow with abrupt transitions; limited cohesion in ideas. Ideas feel disconnected and transitions are often abrupt, making it harder to follow.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eClearer progression of ideas with logical transitions; cohesive flow throughout. Transitions and relationships between ideas are clear, making the argument easy to follow.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eCohesion\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eLimited range of cohesive devices (e.g., simple conjunctions like \u0026ldquo;and,\u0026rdquo; \u0026ldquo;but\u0026rdquo;); repetitive and basic connectors. Results in a monotonous reading experience.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eVaried cohesive devices (e.g., \u0026ldquo;furthermore,\u0026rdquo; \u0026ldquo;in addition\u0026rdquo;); enriched readability and fluidity in connections. Diverse devices enhance text flow and create a more engaging experience.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eArgument Structure\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eSimple, linear argument structure with basic topic sentences; minimal engagement with counterarguments. Arguments are often one-dimensional, lacking depth and multiple perspectives.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eComplex argument structure, integrating counterarguments and multiple perspectives; supports claims with depth. Argument is multifaceted and enriched with examples and counterpoints, showcasing critical thinking.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eLexical Resource\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eLimited vocabulary, often with vague phrasing; struggles with precise expression (e.g., \u0026ldquo;good for the economy\u0026rdquo;). Vocabulary lacks specificity which makes ideas sound general or imprecise.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eWider vocabulary with more specific phrasing; articulates complex ideas (e.g., \u0026ldquo;stimulate innovation and dynamism\u0026rdquo;). Vocabulary is precise which allows for more accurate expression of complex concepts.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eCommon Errors\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eMore frequent, including vague language, simplistic vocabulary, and underdeveloped points. Errors often detract from the clarity and persuasiveness of the essay.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eFewer errors; demonstrates precise language, appropriate vocabulary, and depth in elaboration. Language is accurate and well-elaborated, contributing to a polished, coherent essay.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"},{"header":"Discussion","content":"\u003cp\u003eThe findings of this study illuminate the differences between the discourse features, content organization, and argumentation styles of ChatGPT-generated IELTS Writing Task 2 essays, revealing how these elements correlate with band scores of 6 and 7. By investigating these characteristics, this study uncovers not only the intricacies of effective writing but also the broader implications for language education in the context of emerging AI technologies.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCoherence and Cohesion: A Philosophical Perspective\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe critical role of coherence in writing aligns with the philosophical notion of clarity in communication. Coherence serves as the backbone of effective discourse; it shapes the reader\u0026apos;s journey through the text. Band 6 essays often faltered in this regard, presenting a rudimentary logical flow that hindered understanding. This observation echoes the work of scholars likeCrossley et al. (2016), who argue that coherence is vital for producing high-quality writing. The findings suggest that learners producing band 6 essays might be grappling with a lack of engagement with their material, a point made by Schmid (2020), who emphasizes the importance of narrative coherence in fostering understanding.\u003c/p\u003e\n\u003cp\u003eIn contrast, band 7 essays exhibited a sophisticated use of coherence and cohesion, employing a range of cohesive devices that enriched the narrative. This aligns with Hyland \u0026amp; Jiang (2016)\u0026rsquo;s assertion that effective academic writing relies on the strategic use of cohesion to guide readers through complex arguments. The significant difference in the application of cohesive devices underscores the necessity for writing instruction to prioritize these aspects, promoting an awareness of how language constructs meaning (Zahra et al., 2021)\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eArgument Structure: Engaging with Complexity\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe examination of argument structures highlights the importance of depth in writing, reflecting the philosophical underpinnings of critical thinking and engagement with complexity. Band 6 essays often presented superficial arguments, failing to explore topics beyond a surface level. This finding resonates with the work ofTarchi et al. (2023), who emphasize that proficient writers engage in a recursive process of planning, translating, and reviewing their arguments. The lack of in-depth analysis in band 6 essays suggests a disconnect between the writer and the subject matter, hindering their ability to engage critically with diverse perspectives.\u003c/p\u003e\n\u003cp\u003eConversely, band 7 essays showcased a more intricate argument structure, incorporating counterarguments and perspectives. This complexity reflects the findings of a growing body of literature emphasizing the necessity of critical engagement in writing(Mao \u0026amp; Lee, 2023). By presenting multiple viewpoints, band 7 essays demonstrated an understanding of the multifaceted nature of real-world issues, inviting readers to consider the implications of various arguments. This underscores the importance of fostering a critical mindset in learners, encouraging them to explore and articulate diverse perspectives in their writing.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLexical Resource: The Power of Language\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe analysis of lexical choices between band scores reveals the profound impact of vocabulary on writing quality. Band 6 essays, characterized by vague phrasing and limited vocabulary, underscore the philosophical significance of language as a tool for expression. The work of Halliday (1985) emphasizes that language serves not only as a medium of communication but also as a means of constructing reality. The limited lexical resources observed in band 6 essays suggest a constricted worldview, which ultimately restricts the writer\u0026rsquo;s ability to convey complex ideas effectively.\u003c/p\u003e\n\u003cp\u003eIn contrast, band 7 essays demonstrated a more sophisticated command of language, employing varied vocabulary that enriched their arguments. This finding aligns with the assertions of Cazden (1988), who posits that a robust lexical resource enables writers to articulate their thoughts with precision. The disparity in vocabulary usage between the two bands suggests that educational practices must prioritize vocabulary development to foster a richer understanding of language and its expressive potential.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSignificance and Contribution of the Study\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study contributes to the growing body of literature on AI and writing education by providing empirical evidence on the discourse features of AI-generated texts and their correlation with established writing proficiency standards. By analyzing the distinct characteristics of essays produced by ChatGPT, this research not only enriches our understanding of writing proficiency but also highlights the potential of AI in shaping language learning practices.\u003c/p\u003e\n\u003cp\u003eMoreover, the findings underscore the importance of refining writing instruction to address the complexities of academic discourse. By emphasizing coherence, cohesive devices, argument structure, and lexical resource, educators can better equip learners with the skills necessary to navigate the challenges of academic writing. The implications of this study extend beyond IELTS preparation, offering valuable insights into the nature of effective communication in an increasingly digital landscape.\u003c/p\u003e\n\u003cp\u003eThrough the lens of discourse analysis, this study\u0026rsquo;s findings reveal key differences in the discourse features that mark varying levels of writing proficiency. By focusing on coherence, cohesion, argument structure, and lexical resource, we observe how AI-generated texts aim to replicate human discourse but often miss elements requiring deeper contextual understanding and authentic expression. This suggests that while AI can simulate certain language patterns, it lacks the situational awareness that human discourse naturally conveys.\u003c/p\u003e\n\u003cp\u003eFurthermore, the distinctions between Band 6 and Band 7 essays go beyond surface features and reflect a varying engagement with language conventions specific to academic and high-stakes testing environments like IELTS. This reinforces the idea that discourse is influenced by communicative purpose and audience awareness. Teachers can leverage discourse analysis to help learners identify and apply effective discourse strategies, aiding them in understanding what is valued in academic writing. This study contributes to ongoing research into how discourse analysis can inform the development and assessment of AI-generated language, offering insights into both the potential and the limits of AI as a tool in discourse education.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThis study has provided a comprehensive analysis of the discourse features and writing proficiency levels of ChatGPT-generated IELTS Writing Task 2 essays, revealing distinct patterns that align with the band descriptors for scores 6 and 7. The findings highlight the importance of coherence, cohesive devices, argument structure, and lexical resource in determining writing quality. Band 6 essays exhibited limitations in these areas, often presenting disjointed ideas and simplistic vocabulary, whereas band 7 essays demonstrated a more sophisticated command of language, showcasing complex argumentation and effective cohesion.\u003c/p\u003e\n\u003cp\u003eThe implications of this research extend beyond IELTS preparation, suggesting that AI-generated texts can serve as valuable tools for teaching writing. By focusing on the identified discourse features, educators can develop targeted instructional strategies to improve writing proficiency among learners. The integration of AI in language education presents an opportunity for personalized learning experiences, allowing students to engage critically with their writing processes and receive real-time feedback.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eFuture Studies\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWhile this study offers valuable insights, it is not without limitations. First, the analysis is based solely on AI-generated texts, which may not fully represent the writing capabilities of human learners. Future research could explore the comparative analysis of AI-generated essays and those produced by actual students, providing a more holistic understanding of writing proficiency.\u003c/p\u003e\n\u003cp\u003eSecond, the study focused exclusively on IELTS Writing Task 2 essays, limiting the generalizability of the findings to other writing contexts or genres. Future investigations could examine different writing tasks across various academic disciplines to further validate the identified discourse features and their implications for writing instruction. Moreover, the study did not account for the potential variability in AI output based on different prompts or contexts, which may influence the discourse features and writing quality. Future research could employ a wider range of prompts to assess how variations in input impact the characteristics of AI-generated texts.\u003c/p\u003e\n\u003cp\u003eFinally, exploring the perceptions of educators and learners regarding the use of AI-generated texts in writing education would provide valuable insights into the practical applications and challenges of integrating AI into language instruction. Understanding how these stakeholders perceive AI\u0026apos;s role in writing can guide future pedagogical approaches and enhance the effectiveness of writing instruction in an increasingly digital age.\u003c/p\u003e\n\u003cp\u003eIn conclusion, this study opens avenues for further exploration into the intersection of AI and writing education, emphasizing the need for ongoing research to refine pedagogical practices and equip learners with the skills necessary to navigate the complexities of academic discourse in the 21st century.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNo funds, grants, or other support was received.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for Publication\u003c/strong\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAll participants involved in this study provided their consent for the publication of their data and findings.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData cannot be shared openly to protect study participant privacy. However, it is available and will be shared upon request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eClinical trial number\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Not applicable\u003c/p\u003e\n\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\n\u003cp\u003eThe author, Pourya Javahery, solely conceptualized, researched, and wrote the entire article, including the analysis and interpretation of data, drafting, and revising the manuscript.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eAhmad, Z. (2019). Analyzing Argumentative Essay as an Academic Genre on Assessment Framework of IELTS and TOEFL. In: Hidri, S. (eds) English Language Teaching Research in the Middle East and North Africa. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-319-98533-6_13\u003c/li\u003e\n\u003cli\u003eArzhadeeva, D., \u0026amp; Kudinova, N. (2020). Using Debate to develop Writing Skills for IELTS Writing Task 2 among STEM Students. \u003cem\u003eJournal of Language and Education\u003c/em\u003e, \u003cem\u003e6\u003c/em\u003e(4), 30\u0026ndash;43. https://doi.org/10.17323/jle.2020.10424\u003c/li\u003e\n\u003cli\u003eBurstein, J., Marcu, D., \u0026amp; Knight, K. (2003). Finding the WRITE stuff: automatic identification of discourse structure in student essays. \u003cem\u003eIEEE Intelligent Systems\u003c/em\u003e, \u003cem\u003e18\u003c/em\u003e(1), 32\u0026ndash;39. https://doi.org/10.1109/mis.2003.1179191\u003c/li\u003e\n\u003cli\u003eConnor, U., \u0026amp; Mbaye, A. (2002). 14. DISCOURSE APPROACHES TO WRITING ASSESSMENT. \u003cem\u003eAnnual Review of Applied Linguistics\u003c/em\u003e, \u003cem\u003e22\u003c/em\u003e, 263\u0026ndash;278. https://doi.org/10.1017/s0267190502000144\u003c/li\u003e\n\u003cli\u003eCrossley, S. A., Kyle, K., \u0026amp; McNamara, D. S. (2016). The development and use of cohesive devices in L2 writing and their relations to judgments of essay quality. \u003cem\u003eJournal of Second Language Writing\u003c/em\u003e, \u003cem\u003e32\u003c/em\u003e, 1\u0026ndash;16. https://doi.org/10.1016/j.jslw.2016.01.003\u003c/li\u003e\n\u003cli\u003eCrossley, S. A., \u0026amp; McNamara, D. S. (2016). Say more and be more coherent: How text elaboration and cohesion can increase writing quality. \u003cem\u003eJournal of Writing Research\u003c/em\u003e, \u003cem\u003e7\u003c/em\u003e(3), 351\u0026ndash;370. https://doi.org/10.17239/jowr-2016.07.03.02\u003c/li\u003e\n\u003cli\u003eEscalante, J., Pack, A., \u0026amp; Barrett, A. (2023). AI-generated feedback on writing: insights into efficacy and ENL student preference. \u003cem\u003eInternational Journal of Educational Technology in Higher Education\u003c/em\u003e, \u003cem\u003e20\u003c/em\u003e(1). https://doi.org/10.1186/s41239-023-00425-2\u003c/li\u003e\n\u003cli\u003eGasaymeh, A. M., Beirat, M. A., \u0026amp; Qbeita, A. a. A. (2024). University students\u0026rsquo; insights of Generative Artificial Intelligence (AI) writing tools. \u003cem\u003eEducation Sciences\u003c/em\u003e, \u003cem\u003e14\u003c/em\u003e(10), 1062. https://doi.org/10.3390/educsci14101062\u003c/li\u003e\n\u003cli\u003eGuo, K., \u0026amp; Li, D. (2024). Understanding EFL students\u0026rsquo; use of self-made AI chatbots as personalized writing assistance tools: A mixed methods study. \u003cem\u003eSystem\u003c/em\u003e, \u003cem\u003e124\u003c/em\u003e, 103362. https://doi.org/10.1016/j.system.2024.103362\u003c/li\u003e\n\u003cli\u003eHazrati, Y. (2023). Content analysis of lexical cohesive devices in the lectures of the Minister of Health, Treatment and Medical Education of the Raisi Government in the field of health care, Treatment and health. \u003cem\u003eInternational Journal of Multicultural and Multireligious Understanding\u003c/em\u003e, \u003cem\u003e10\u003c/em\u003e(12), 439. https://doi.org/10.18415/ijmmu.v10i12.5446\u003c/li\u003e\n\u003cli\u003eHyland, K., \u0026amp; Jiang, F. (2016). Is academic writing becoming more informal? \u003cem\u003eEnglish for Specific Purposes\u003c/em\u003e, \u003cem\u003e45\u003c/em\u003e, 40\u0026ndash;51. https://doi.org/10.1016/j.esp.2016.09.001\u003c/li\u003e\n\u003cli\u003eLiontou, T. (2022). Automated Discourse Analysis Techniques and Implications for Writing Assessment. \u003cem\u003eLanguages\u003c/em\u003e, \u003cem\u003e8\u003c/em\u003e(1), 3. https://doi.org/10.3390/languages8010003\u003c/li\u003e\n\u003cli\u003eLivberber, T., \u0026amp; Ayvaz, S. (2023). The impact of Artificial Intelligence in academia: Views of Turkish academics on ChatGPT. \u003cem\u003eHeliyon\u003c/em\u003e, \u003cem\u003e9\u003c/em\u003e(9), e19688. https://doi.org/10.1016/j.heliyon.2023.e19688\u003c/li\u003e\n\u003cli\u003eMarkauskaite, L., Marrone, R., Poquet, O., Knight, S., Martinez-Maldonado, R., Howard, S., Tondeur, J., De Laat, M., Shum, S. B., Ga\u0026scaron;ević, D., \u0026amp; Siemens, G. (2022). Rethinking the entwinement between artificial intelligence and human learning: What capabilities do learners need for a world with AI? \u003cem\u003eComputers and Education Artificial Intelligence\u003c/em\u003e, \u003cem\u003e3\u003c/em\u003e, 100056. https://doi.org/10.1016/j.caeai.2022.100056\u003c/li\u003e\n\u003cli\u003eMarzuki, Widiati, U., Rusdin, D., Darwin, \u0026amp; Indrawati, I. (2023). The impact of AI writing tools on the content and organization of students\u0026rsquo; writing: EFL teachers\u0026rsquo; perspective. \u003cem\u003eCogent Education\u003c/em\u003e, \u003cem\u003e10\u003c/em\u003e(2). https://doi.org/10.1080/2331186x.2023.2236469\u003c/li\u003e\n\u003cli\u003eNesi, H., \u0026amp; Gardner, S. (2012). \u003cem\u003eGenres across the disciplines: Student Writing in Higher Education\u003c/em\u003e. Cambridge University Press.\u003c/li\u003e\n\u003cli\u003eNguyen, H. T. M., \u0026amp; Van Anh Le, N. (2024). Text Complexity of Cambridge-delivered IELTS Academic Reading Tests: Comparability with IELTS Academic Reading Practice Tests from Other Publishers. \u003cem\u003eTeaching English as a Second or Foreign Language--TESL-EJ\u003c/em\u003e, \u003cem\u003e28\u003c/em\u003e(2). https://doi.org/10.55593/ej.28110a4\u003c/li\u003e\n\u003cli\u003eNguyen, L.Q., Le, H.V. Improving L2 learners\u0026rsquo; IELTS task 2 writing: the role of model essays and noticing hypothesis. \u003cem\u003eLang Test Asia\u003c/em\u003e \u003cstrong\u003e12\u003c/strong\u003e, 58 (2022). https://doi.org/10.1186/s40468-022-00206-0\u003c/li\u003e\n\u003cli\u003eSchmid, H. (2020). \u003cem\u003eThe dynamics of the linguistic system: Usage, Conventionalization, and Entrenchment\u003c/em\u003e. Oxford University Press.\u003c/li\u003e\n\u003cli\u003eSpence, L. K. (2010). Discerning Writing Assessment: Insights into an Analytical Rubric. \u003cem\u003eLanguage Arts\u003c/em\u003e, \u003cem\u003e87\u003c/em\u003e(5), 337\u0026ndash;352. https://doi.org/10.58680/la201010535\u003c/li\u003e\n\u003cli\u003eTeng, M. F. (2024a). A Systematic Review of ChatGPT for English as a Foreign Language Writing: Opportunities, Challenges, and Recommendations. \u003cem\u003eInternational Journal of TESOL Studies\u003c/em\u003e, \u003cem\u003e6\u003c/em\u003e(3).\u003c/li\u003e\n\u003cli\u003eTeng, M. F. (2024b). \u0026ldquo;ChatGPT is the companion, not enemies\u0026rdquo;: EFL learners\u0026rsquo; perceptions and experiences in using ChatGPT for feedback in writing. \u003cem\u003eComputers and Education: Artificial Intelligence\u003c/em\u003e, \u003cem\u003e7\u003c/em\u003e, 100270. \u003c/li\u003e\n\u003cli\u003eTseng, W., \u0026amp; Warschauer, M. (2023). AI-writing tools in education: if you can\u0026rsquo;t beat them, join them. \u003cem\u003eJournal of China Computer-Assisted Language Learning\u003c/em\u003e, \u003cem\u003e3\u003c/em\u003e(2), 258\u0026ndash;262. https://doi.org/10.1515/jccall-2023-0008\u003c/li\u003e\n\u003cli\u003eVan Dijk, T. A. (2010). \u003cem\u003eDiscourse and context: A Sociocognitive Approach\u003c/em\u003e. Cambridge University Press.\u003c/li\u003e\n\u003cli\u003eVincheh, M. H., Mirzaei, A., \u0026amp; Roohani, A. (2024). A cognitive diagnostic approach to IELTS speaking test: unveiling the subskills and test-takers\u0026rsquo; perceptions. \u003cem\u003eLanguage Testing in Asia\u003c/em\u003e, 14(1). https://doi.org/10.1186/s40468-024-00311-2\u003c/li\u003e\n\u003cli\u003eWang, Y., \u0026amp; Xie, Q. (2022). Diagnostic assessment of novice EFL learners\u0026rsquo; discourse competence in academic writing: a case study. \u003cem\u003eLanguage Testing in Asia\u003c/em\u003e, \u003cem\u003e12\u003c/em\u003e(1). https://doi.org/10.1186/s40468-022-00197-y\u003c/li\u003e\n\u003cli\u003eWarschauer, M., Tseng, W., Yim, S., Webster, T., Jacob, S., Du, Q., \u0026amp; Tate, T. (2023b). The affordances and contradictions of AI-generated text for writers of English as a second or foreign language. \u003cem\u003eJournal of Second Language Writing\u003c/em\u003e, \u003cem\u003e62\u003c/em\u003e, 101071. https://doi.org/10.1016/j.jslw.2023.101071\u003c/li\u003e\n\u003cli\u003eYao, S. (2014). An analysis of Chinese students\u0026rsquo; performance in IELTS academic writing. \u003cem\u003eThe New English Teacher\u003c/em\u003e, \u003cem\u003e8\u003c/em\u003e(2). http://www.assumptionjournal.au.edu/index.php/newEnglishTeacher/article/download/295/253\u003c/li\u003e\n\u003cli\u003eZahra, G. M., Emilia, E., \u0026amp; Nurlaelawati, I. (2021). An analysis of cohesion and coherence of descriptive texts written by junior high school students. \u003cem\u003eAdvances in Social Science, Education and Humanities Research/Advances in Social Science, Education and Humanities Research\u003c/em\u003e. https://doi.org/10.2991/assehr.k.210427.030\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"AI-generated texts, IELTS Writing Task 2, discourse analysis, writing proficiency, pedagogy","lastPublishedDoi":"10.21203/rs.3.rs-5608928/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5608928/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis study examines the discourse features of ChatGPT-generated IELTS Writing Task 2 essays, with a focus on the specific characteristics associated with band descriptors for scores of 6 and 7.Utilizing discourse and content analyses approaches, the studyexamines coherence, cohesion, argument structure, and lexical resource to understand the discourse characteristics indicative of different proficiency levels. Findings reveal that band 6 essays exhibit basic coherence with abrupt transitions and limited use of cohesive devices which result in a more linear argument structure and simpler lexical choices. Conversely, band 7 essays demonstrate a clearer progression of ideas, enhanced cohesion through varied cohesive devices, and a more complex argument structure that effectively integrates counterarguments and depth of analysis. These discourse patterns underscore the potential for AI-generated texts to model proficiency levels in writing and serve as pedagogical tools to improve learner outcomes. By highlighting the discourse elements critical to achieving higher band scores, this study contributes valuable insights into AI\u0026rsquo;s role in supporting language learning and academic writing proficiency.\u003c/p\u003e","manuscriptTitle":"Analyzing Discourse Patterns in ChatGPT-Generated IELTS Writing Task 2 Essays: A Discourse Analysis Approach","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-01-01 05:25:45","doi":"10.21203/rs.3.rs-5608928/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2bd18680-7954-4e0c-bd31-a2294823fd0c","owner":[],"postedDate":"January 1st, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-01-01T10:23:25+00:00","versionOfRecord":[],"versionCreatedAt":"2025-01-01 05:25:45","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5608928","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5608928","identity":"rs-5608928","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}