Developing Meaning-Based Complexity Indices Based on Cognitive-Functional Connectives and Validating Their Predictive Utility in Automated Writing Evaluation

Developing Meaning-Based Complexity Indices Based on Cognitive-Functional Connectives and Validating Their Predictive Utility in Automated Writing Evaluation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Developing Meaning-Based Complexity Indices Based on Cognitive-Functional Connectives and Validating Their Predictive Utility in Automated Writing Evaluation Nahyung Kong This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7247858/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract This study develops and validates three meaning-based complexity indices— Academic Diversity (ADiv), Academic Weight (AWeight), and Academic Rate (ARate) —to quantify academic vocabulary usage in native Korean undergraduate students’ argumentative writing and assess their predictive value in writing evaluation. Drawing on a large-scale national writing assessment corpus comprising 2,429 essays from two academic prompts, the study investigates how these indices reflect cognitive depth and discourse competence in L1 academic writing. Regression analyses reveal that AWeight , which captures the semantic sophistication of academic terms based on graded difficulty, exhibits a consistently positive relationship with writing scores. Contrary to prior assumptions of an inverted-U shape, no detrimental effects of “excessive complexity” were observed within the actual learner range. AWeight also emerged as the most effective diagnostic indicator for distinguishing high- and low-scoring writers, with a potential benchmark range identified between 20–25 points. While integrated models incorporating all three indices yielded only marginal improvements in predictive power (R² = 0.0574), clustering analyses revealed distinct lexical strategy types , highlighting variation in how native speakers deploy academic vocabulary. Notably, a subgroup exhibiting “inappropriate complexity”—high lexical sophistication but low scores—suggests that lexical complexity alone is insufficient without discourse-level alignment. These findings underscore the gatekeeping function of academic vocabulary complexity in automated writing evaluation (AWE) systems and call for multi-dimensional assessment frameworks that integrate semantic depth, discourse appropriateness, and strategic deployment . The study contributes theoretically by reframing lexical complexity as a context-sensitive, functional construct , and practically by proposing pedagogically interpretable indices for AWE and learner diagnostics in native academic writing contexts. academic writing argumentative writing academic vocabulary lexical complexity native Korean writers automated writing evaluation argumentative discourse Figures Figure 1 Figure 2 Figure 3 Figure 4 1. Introduction This study aims to empirically investigate the relationship between academic vocabulary (AV) complexity and writing assessment scores, while also exploring the potential of integrated, multi-indicator predictive models. With the advancement of Automated Writing Evaluation (AWE) systems, there has been growing interest in quantifying linguistic complexity in student writing. However, early studies have primarily relied on surface-level features such as sentence length, word frequency, and syntactic complexity (Kyle & Crossley, 2015 ; Lu, 2012 ), which have been consistently criticized for their inability to capture writers' cognitive depth and academic reasoning. In the Korean educational context, where academic writing serves as a critical gatekeeping mechanism for university advancement and scholarly participation, the strategic deployment of academic vocabulary represents more than linguistic competency—it embodies cultural and cognitive dimensions of scholarly discourse deeply embedded in East Asian educational traditions. Unlike Western academic writing that often emphasizes clarity and directness, Korean scholarly discourse values lexical sophistication and conceptual density as markers of intellectual maturity and cultural literacy. This cultural specificity necessitates assessment frameworks that honor indigenous scholarly conventions while maintaining methodological rigor. To successfully participate in academic discourse communities, writers must effectively employ field-specific terminology, articulate abstract concepts with precision, and structure logical reasoning within coherent discourse frameworks (Biber & Gray, 2010 ; Corson, 1997 ). These abilities are not adequately represented by simple metrics like word rarity or length, necessitating meaning-based complexity indices that better reflect cognitive and discourse-level sophistication. In particular, academic vocabulary—also referred to in the Korean context as 사고도구어 (thinking tool words)—has become increasingly essential in university-level writing instruction , where academic literacy is emphasized across disciplines. The Korean conceptualization of 사고도구어 extends beyond English-based academic word lists to encompass culturally grounded cognitive tools that facilitate abstract reasoning and scholarly argumentation within Confucian intellectual traditions. In higher education, writing is not only a communicative act but also a cognitive tool for constructing disciplinary knowledge and demonstrating epistemic engagement. Accordingly, students are expected to utilize abstract, metalinguistic expressions to connect, justify, contrast, or condition arguments—functions that are central to both academic persuasion and conceptual organization. This focus on Korean L1 writers addresses a significant gap in Asian language testing research, where most complexity studies have focused on English L2 writing. By examining native speaker strategic variation in academic vocabulary deployment, this study contributes to understanding how cultural and linguistic factors shape lexical complexity patterns in ways that transcend simple L1/L2 distinctions. This study thus focuses on academic vocabulary as a core construct in developing lexical complexity indices that reflect native undergraduate writers' capacity for conceptual development and discourse organization. By operationalizing these meaning-oriented indicators, the study aims to capture cognitively loaded vocabulary patterns that underlie successful academic writing within the distinctive cultural and educational context of Korean higher education. Accordingly, this study addresses the following research questions: RQ1. What is the relationship between academic vocabulary complexity and writing assessment scores? RQ1-1. Does the academic vocabulary sophistication index (AWeight) exhibit a non-linear (e.g., inverted U-shaped) relationship with writing scores? RQ1-2. What is the optimal range of AWeight that contributes positively to writing scores, and what effects emerge when this range is exceeded? RQ2. Does a composite model integrating academic vocabulary diversity (ADiv), sophistication (AWeight), and density (ARate) provide complementary insights and improve the interpretability of writing assessment compared to single-indicator models? RQ2-1. What interpretive value or diagnostic potential does the integration of multiple indicators offer beyond predictive accuracy? RQ2-2. How do different indicators contribute to model explanations, and which combinations yield the most informative representations of writing performance? RQ3. How can academic vocabulary complexity indices help distinguish writing strategies between high-scoring and low-scoring groups? RQ3-1. What distributional patterns or strategic tendencies are observed in the use of AV indices across top and bottom scoring groups? RQ3-2. What general patterns or characteristics can be identified in cases where high AV complexity coincides with low writing scores? 2. Literature Review: Academic Vocabulary and Automated Writing Assessment Academic vocabulary refers to a set of lexical resources that are essential for higher-order thinking and academic achievement, particularly in formal and scholarly contexts (Martin, 1976 ; Nation, 2001 ). Unlike basic conversational vocabulary or domain-specific technical terms, academic vocabulary comprises cognitively functional expressions that facilitate complex meaning-making across disciplines. These lexical items serve to define abstract concepts, organize logical relations, and construct coherent academic discourse. One of the key characteristics of academic vocabulary lies in its abstract and metacognitive nature. Rather than reflecting surface-level lexical difficulty, these items function as tools for conceptualization, classification, analysis, comparison, and justification (Corson, 1997 ; Coxhead, 2000 ). Accordingly, academic vocabulary plays a crucial role in shaping argumentation and reasoning processes, making it a core component of academic literacy. For native speakers, academic vocabulary is not merely a communicative tool but a vehicle for constructing and manipulating abstract knowledge. It has been classified as both "academic" and "cognitive" vocabulary and is distinguished from field-specific jargon by its cross-disciplinary utility (Schmitt & Schmitt, 2020 ). Its role in internalizing and generating academic discourse structures imposes substantial cognitive demands—even on native speakers with limited academic literacy (Laufer, 2013 ). This study specifically focuses on native Korean undergraduate writers, whose ability to use academic vocabulary reflects their discourse-level academic competence, rather than second-language acquisition. In academic writing, the use of academic vocabulary is closely tied to writing competence. These expressions serve not only as lexical choices but also as strategic devices for constructing claims, signaling reasoning structures, and conveying epistemic stance. Thus, the ability to deploy academic vocabulary effectively is indicative of a writer’s capacity for organizing arguments, engaging critically with content, and producing coherent academic discourse—core dimensions of university-level writing performance. Despite its importance, academic vocabulary has received limited attention in writing assessment research. Traditional studies have relied on surface-level lexical complexity measures such as the Type-Token Ratio (TTR), Mean Segmental TTR (MSTTR), lexical diversity, and rare word usage rates (Kyle & Crossley, 2015 ; Lu, 2012 ). While such indicators are useful for gauging linguistic fluency or lexical range, they fail to capture meaning-oriented complexity—such as discourse function, semantic load, and strategic deployment within a text (Biber & Gray, 2010 ). In response to these limitations, recent studies have begun to focus more on the functional roles of vocabulary within discourse rather than on frequency-based metrics. Academic vocabulary has emerged as a central concept in this shift, with empirical research demonstrating its predictive power for reading comprehension, critical thinking, and academic success (Libben et al., 2007). Nonetheless, in the domain of Automated Writing Evaluation (AWE), research quantifying the strategic use of academic vocabulary remains scarce. Most current approaches are limited to detecting basic connectives or calculating lexical diversity in a surface-level manner. As AWE systems continue to evolve, there is a growing need to integrate more cognitively and rhetorically grounded scoring features. However, most existing work has emphasized the educational significance of academic vocabulary or its conceptual classification, without sufficiently addressing its operationalization within automated assessment models. This study addresses that gap by proposing meaning-based complexity indices that capture how academic vocabulary is deployed to construct conceptual meaning and discourse structure. By incorporating such indices into AWE systems, writing assessment can move beyond structural surface features toward evaluating deeper cognitive and rhetorical dimensions. These indicators can support high-level functions such as identifying high-performing writers, diagnosing ineffective lexical strategies, and providing targeted feedback on discourse construction. This approach holds particular promise for profiling native writers’ strategic variation in academic vocabulary use —a dimension often overlooked in current automated scoring frameworks. 3. Research Design 3.1. Data and Writing Tasks This study analyzed two argumentative writing tasks from the 2023 National Corpus for the Development of Korean Writing Proficiency Assessment , specifically the “Everyone's Corpus(모두의 말뭉치)” section compiled by the National Institute of the Korean Language (NIKL). Specifically, we used data from the “Everyone's Corpus(모두의 말뭉치)” section of the 2023 dataset, which contains a total of 5,000 student-written texts. For this study, 2,429 essays were analyzed: 1,231 responses to Task Q2 (on legal regulations of hate speech) and 1,198 responses to Task Q3 (on restrictions on elderly drivers’ licenses). The corpus consists of argumentative texts written by native speakers of Korean enrolled in undergraduate academic writing courses at university-level institutions in Korea. The participants were all native Korean speakers, primarily first-year students, who were participating in regular curricular writing instruction. The writing tasks were administered as part of official coursework during academic semesters, under standardized test-like conditions (90-minute limit, no access to external resources). These constraints ensured that the essays reflect students’ authentic academic writing competence under evaluative pressure. This controlled setting ensures that the resulting texts reflect learners’ authentic academic writing competence within evaluative constraints. Task Q2 required students to write a logical argument regarding whether hate speech should be legally regulated. The task prompt was as follows: Hate speech refers to verbal or non-verbal acts of public insult, demeaning, contempt, threat, or incitement to discrimination and violence against individuals or groups based on unjustifiable hatred. As hate speech denies the dignity of targeted individuals or groups and has detrimental effects on society, there are increasing calls for its legal regulation. Present your opinion on whether hate speech should be legally regulated, and support your view with logical reasoning. [Instructions] Write a complete essay with an introduction, body, and conclusion. Do not include a title. Length: Approximately 1,000 characters (± 200, including spaces). Time limit: 90 minutes. Task Q3 asked students to present their opinions on whether elderly drivers should have their licenses restricted. The task prompt was: In Korea, anyone aged 18 or older can obtain a driver’s license. However, due to recent traffic accidents caused by elderly drivers' slow responses, there are increasing arguments that people should be required to relinquish their licenses at a certain age. As Korean society continues to age, this issue can no longer be ignored. Present your opinion on whether driver’s licenses for elderly individuals should be restricted, and support your view with logical reasoning. [Instructions] Write a complete essay with an introduction, body, and conclusion. Do not include a title. Length: Approximately 1,000 characters (± 200, including spaces). Time limit: 90 minutes. The two tasks share structural characteristics that make them suitable for comparative analysis: they are both argumentative essays with similar length and time constraints, employ the same evaluation criteria (content, organization, and language use), and explicitly require an introduction–body–conclusion format. However, the nature of the topics provides a meaningful contrast for exploring academic vocabulary strategies: Task Q2 involves moral and social value judgments, while Task Q3 requires more policy-oriented and pragmatic reasoning. All texts were provided in JSON format and include individual scores for three dimensions— content (eva_score_con), organization (eva_score_org), and language use (eva_score_exp)—as well as a total score. This study used the final evaluator’s scores as the basis for analysis. Among the evaluated domains, the language use score, which assesses lexical appropriateness, style, and grammatical accuracy, was considered most closely related to the use of academic vocabulary. 3.2. Independent Variables: Academic Vocabulary Complexity Indicators This study established three complexity indicators as independent variables to quantitatively measure the usage patterns of academic vocabulary , which serves as a core lexical resource reflecting writers’ cognitive competence in academic discourse. The academic vocabulary list was initially derived from Shin ( 2004 ) and was reclassified by the researcher based on discourse functions (see Kong, 2025 ). The classification criteria included cognitive functions such as causation, justification, condition, contrast , and exemplification , and each lexical item was graded according to its functional role and level of difficulty. The first indicator, Academic Diversity (ADiv) , refers to the number of unique academic vocabulary types (i.e., type count) that appear in a text without repetition. ADiv captures the breadth of a writer’s academic lexical repertoire, reflecting their ability to strategically activate a range of academic concepts beyond mere lexical variety. It goes beyond surface-level counts to indirectly measure a writer’s ability to strategically activate a diverse range of academic concepts. A high ADiv value implies the use of varied thinking strategies and conceptual tools and is positively associated with advanced discourse organization abilities (Biber & Gray, 2010 ). The second indicator, Academic Weight (AWeight) , is calculated by multiplying each academic term’s frequency by its assigned cognitive weight, derived from a four-level difficulty scale within the text. The grading system follows the four-level scheme proposed in Shin ( 2004 ), ranging from Grade 1 (basic conceptual terms) to Grade 4 (highly abstract conceptual terms). This indicator aims to reflect not merely the amount but the conceptual complexity and abstractness of the vocabulary used. As such, AWeight serves as a proxy for cognitive depth , capturing nuances often missed by frequency-based metrics. It also assesses the sophistication and precision of vocabulary usage, which underpin academic persuasiveness. The third indicator, Academic Rate (ARate) , is the proportion (%) of academic vocabulary tokens out of the total tokens in a text. It indicates the density of academic vocabulary usage and can serve as a proxy for the overall “academicness” of the writing. While ARate helps assess the extent of academic discourse tendencies in learners’ language, previous studies (e.g., Hinkel, 2003; Norris & Ortega, 2009) caution that excessive density may reduce readability and clarity. Accordingly, ARate can also be useful for detecting overuse or inappropriate complexity . Together, these three indicators provide complementary perspectives on academic vocabulary usage and offer a multi-dimensional basis for assessing the lexical complexity of learner texts. Although both AWeight and ARate are positively correlated, they capture distinct dimensions of academic vocabulary use. AWeight emphasizes the semantic weight and cognitive sophistication of each vocabulary item, assigning graded difficulty based on its abstractness and functional role. In contrast, ARate measures the proportional density of academic vocabulary relative to the total token count, providing a coarse-grained indicator of overall lexical register. Thus, AWeight represents qualitative complexity—highlighting the strategic use of cognitively demanding terms—whereas ARate denotes quantitative saturation of academic vocabulary within a text. Distinguishing these two dimensions is essential for a nuanced interpretation of lexical complexity. 3.3. Analysis Procedures 3.3.1. Variables and Statistical Assumptions Dependent Variables The dependent variables were the total writing score and the expression score (eva_score_exp) provided by the National Institute of Korean Language. The expression score reflects vocabulary appropriateness, grammatical accuracy, and stylistic coherence—dimensions directly related to academic vocabulary usage. The total score, aggregating content, organization, and expression components, served as the primary indicator of overall writing quality for AWE model evaluation. Statistical Assumptions and Preprocessing Prior to analysis, we examined distributional properties and statistical assumptions: Normality: Shapiro-Wilk tests and Q-Q plots confirmed approximate normality for dependent variables Multicollinearity: Variance Inflation Factor (VIF) analysis revealed high correlation between AWeight and ARate (r = 0.92, VIF > 5), necessitating careful interpretation in integrated models Outliers: Values exceeding 3 standard deviations were identified but retained after sensitivity analysis showed minimal impact on results Missing data: Complete case analysis was employed (n = 2,429 with no missing values) 3.3.2. Four-Stage Sequential Analysis Design Step 1: Nonlinear Relationship Analysis (RQ1) Objective: Investigate linear vs. nonlinear relationships between academic vocabulary complexity and writing performance. Methods: Model Comparison: Linear regression vs. quadratic polynomial models for each indicator (ADiv, AWeight, ARate) Nonlinearity Testing: Specifically tested inverted U-shaped hypothesis for AWeight using quadratic terms Model Selection: AIC/BIC criteria and nested F-tests for model comparison Validation: 10-fold cross-validation with 80/20 train-test split Visualization: Scatter plots with fitted curves and 95% confidence intervals Statistical Power: Post-hoc power analysis confirmed adequate sample size (n = 2,429) for detecting medium effect sizes (f² ≥ 0.15) with power > 0.80. Step 2: Integrated Model Construction (RQ2-1) Objective: Evaluate predictive validity and complementary power of combined indicators. Model Architecture: Baseline Linear Model: ADiv + AWeight + ARate as main effects Enhanced Nonlinear Model: Main effects + AWeight² quadratic term Regularized Model: Ridge regression (α = 0.1) to address multicollinearity Performance Metrics: Accuracy: RMSE, MAE, R² with 95% confidence intervals Validation: Stratified 10-fold cross-validation with shuffling Stability: Bootstrap resampling (n = 1,000) for coefficient confidence intervals Interpretability: SHAP (SHapley Additive exPlanations) values for feature importance Multicollinearity Management: Given high VIF values, we employed: Ridge regression for coefficient stabilization Principal Component Analysis (PCA) as alternative dimensionality reduction Variance decomposition to isolate unique vs. shared variance contributions Step 3: Diagnostic Profiling and Pattern Discovery (RQ2-2) Objective: Identify learner subtypes through unsupervised clustering of indicator profiles. Clustering Methodology: Algorithm Selection: K-means clustering with elbow method and silhouette analysis for optimal k Distance Metric: Euclidean distance on standardized indicators (z-scores) Validation: Gap statistic and bootstrap clustering stability (Jaccard coefficient > 0.75) Alternative Methods: DBSCAN for density-based clustering comparison Cluster Interpretation: Internal Validation: Within-cluster sum of squares, silhouette coefficients External Validation: ANOVA F-tests for cluster differences in writing scores Effect Sizes: Cohen's d for between-cluster comparisons with Bonferroni correction Pedagogical Mapping: Cluster characteristics mapped to instructional implications Step 4: Strategic Group Analysis (RQ3) Objective: Distinguish writing strategies between performance groups and identify inappropriate complexity patterns. Group Definition: High Performers: Top 25th percentile (n = 607, score ≥ 36.0) Low Performers: Bottom 25th percentile (n = 607, score ≤ 24.0) Inappropriate Complexity: Top 25% AWeight + Bottom 25% total score (n = 106, 4.4%) Statistical Comparisons: Between-group Tests: Independent t-tests with Welch's correction for unequal variances Effect Sizes: Cohen's d with 95% confidence intervals, interpreted using Cohen's (1988) benchmarks Distribution Analysis: Kolmogorov-Smirnov tests for distributional differences Practical Significance: Minimum meaningful difference thresholds based on score ranges Qualitative Integration: Text Selection: Systematic sampling of 20 texts per inappropriate complexity case Discourse Features: Manual coding for lexical-argument alignment, register consistency, semantic precision Inter-rater Reliability: Two independent coders with Cohen's κ > 0.80 for qualitative features 3.3.3. Software and Reproducibility All analyses were conducted in R (v4.3.0) using the packages tidyverse , caret , cluster , SHAP , and ggplot2 . Anonymized learner data and output files (in Excel format) are available via [ https://drive.google.com/drive/folders/15rlRytuDy9xOmq9Ts_fHrAy89P35Qniq?usp=drive_link ] to support transparency and reproducibility. Analysis scripts are available upon request. 4. Results 4.1. Descriptive Statistics and Correlation Analysis: Q2–Q3 Task Comparison This section presents a comparative analysis of academic vocabulary complexity indicators—Academic Diversity (ADiv), Academic Weight (AWeight), and Academic Rate (ARate)—across two argumentative writing tasks: Q2 (“Legal Regulation of Hate Speech”) and Q3 (“Restrictions on Elderly Drivers' Licenses”). A total of 2,429 texts were analyzed (Q2: 1,231; Q3: 1,198), and the results are visualized in Fig. 1 . First, regarding Academic Diversity (Fig. 1 -a), the Q3 task (M = 5.205, SD = 2.362) yielded a significantly higher average than Q2 (M = 4.185, SD = 1.973), with a medium effect size (Cohen’s d = 0.467). The Q3 distribution is clearly shifted to the right, with a wider spread and a maximum value of 15.0 compared to 13.0 in Q2. This suggests that the Q3 prompt elicited a broader conceptual range, likely due to its focus on policy evaluation and real-life implications, which demand diverse academic vocabulary. Second, in terms of Academic Weight (Fig. 1 -b and 1 -e), the mean values were relatively similar (Q2: M = 21.096; Q3: M = 22.301), and the effect size was negligible (d = 0.092). Figure 1 -e shows comparable median AWeight scores across tasks, though Q3 exhibits slightly more variability in the upper quartile and a greater number of outliers. This indicates that while both tasks required cognitively demanding vocabulary, Q3 allowed for greater individual variation in the depth of lexical use. Third, the Academic Rate distribution (Fig. 1 -c) showed a slightly higher mean for Q2 (M = 0.058, SD = 0.032) than for Q3 (M = 0.054, SD = 0.031), with a small effect size (d = 0.125). Notably, Q2 exhibits a more peaked distribution around the mode, suggesting more consistent use of academic vocabulary among writers, possibly due to the abstract nature of the hate speech topic. Fourth, correlation analyses (Fig. 1 -d) revealed that AWeight had the strongest relationship with total writing scores in Q2 (r = 0.282), while this correlation decreased in Q3 (r = 0.162). This finding indicates that conceptually sophisticated vocabulary was more predictive of performance in abstract argumentative contexts. ADiv showed consistent but weaker correlations across tasks (Q2: r = 0.174; Q3: r = 0.159), while ARate correlated slightly more in Q2 (r = 0.194) than Q3 (r = 0.167). Finally, Fig. 1 -f presents the effect sizes for each indicator across tasks. ADiv demonstrated the clearest task-level differentiation (d = 0.467), underscoring its role in capturing topical breadth. In contrast, AWeight and ARate showed minimal effect sizes, suggesting task-invariant patterns in lexical depth and density. These findings highlight that academic vocabulary complexity is not uniformly distributed across writing tasks and that task characteristics—such as topical abstraction and evaluative demand—can shape the deployment and impact of specific lexical strategies. Such variation provides empirical grounding for task-sensitive modeling in automated writing evaluation and supports the development of writer-specific diagnostic profiles that capture individual patterns of academic vocabulary deployment. Building on these insights, these preliminary findings establish the foundation for addressing our research questions by demonstrating that academic vocabulary complexity patterns are context-dependent, which will inform our subsequent analyses of nonlinear relationships (RQ1), integrated modeling approaches (RQ2), and strategic profiling of learner groups (RQ3). 4.2. Relationship Between Academic Weight (AWeight) and Writing Scores Academic vocabulary complexity is regarded as a key factor that reflects the sophistication of academic reasoning in written discourse. This section analyzes the relationship between one of the complexity indicators— Academic Weight (AWeight) —and the overall writing scores, aiming to empirically reveal how lexical complexity operates in the actual assessment process. Departing from a purely correlational approach, we employed linear and non-linear regression models as well as interval-based analyses to explore predictive patterns and identify an optimal range of complexity. 4.2.1. Functional Relationship Between AWeight and Writing Performance RQ 1–1 posits that the relationship between academic vocabulary complexity and writing performance may follow a non-linear curve, potentially exhibiting an inverted-U shape. To examine this hypothesis, both linear (1st-degree) and quadratic (2nd-degree polynomial) regression models were applied, and their predictive power was compared using R² values alongside theoretical peak points. The results are illustrated in Fig. 2 -(a) and (b). Scatterplots and bar charts illustrating the relationship between AWeight (weighted academic vocabulary index) and total score for Q2 and Q3 tasks. Subfigures (a) and (b) display linear and quadratic regression fits, with R² values indicating predictive strength. Subfigures (c) and (d) present mean total scores across AWeight intervals with standard deviation bars, showing a general upward trend for Q2 but a flatter pattern for Q3. For the Q2 task, the linear regression model yielded an R² value of 0.0775, while the quadratic model slightly improved to 0.0785 (ΔR² = +0.0010). Similarly, in the Q3 task, the linear model had an R² of 0.0287 and the quadratic model of 0.0308, showing only a marginal difference (ΔR² = +0.0021) that was not statistically significant (F-test for nested models: Q2: F(1, 1228) = 1.30, p > 0.1; Q3: F(1, 1195) = 2.62, p > 0.1). These findings indicate that although AWeight contributes some explanatory power to score prediction, the added value of a non-linear model remains statistically negligible. As clearly shown in Fig. 2 -(a) and (b), the linear and quadratic regression lines nearly overlap across the observed AWeight range, visually confirming the minimal benefit of nonlinear modeling. Furthermore, the theoretical peak values predicted by the quadratic models were AWeight = 86.08 for Q2 and AWeight = 65.89 for Q3—both exceeding the observed learner range (Q2: 0–71; Q3: 0–75). This suggests potential model overfitting and limits the interpretability of the curve within the current dataset. Within the empirical range, the relationship between AWeight and writing scores appears to be unidirectional and positive, with no observed downturn that would support the inverted-U hypothesis. To supplement this interpretation, we conducted an interval-based analysis (Fig. 2 -(c), (d)). In Q2, learners in the lowest AWeight quartile (average = 4.8) achieved a mean score of 26.8, whereas those in the highest quartile (average = 41.7) achieved 38.2. In Q3, scores rose from 28.2 (AWeight = 5.5) to 33.1 (AWeight = 32.1). Crucially, no score decline was observed in the higher AWeight ranges, providing insufficient evidence for the claim that excessive lexical complexity negatively impacts assessment outcomes. In summary, AWeight demonstrates a meaningful positive linear relationship with writing scores. Within the observed learner range, higher AWeight values consistently correspond to improved performance. While the quadratic models suggest a theoretical peak beyond the observed data, such extrapolated interpretations should be treated with caution. 4.2.2. Optimal Range of AWeight and Pedagogical Implications RQ 1–2 sought to determine an efficient range of AWeight that contributes most effectively to writing performance, offering pedagogical insights for instruction and feedback. To this end, we compared the average AWeight of the top and bottom 25% of scorers. The analysis revealed that high-scoring learners (top 25%) had an average AWeight of 25.8 in Q2 and 25.3 in Q3, suggesting a common complexity level associated with strong performance, regardless of task type. In contrast, the bottom 25% averaged 17.2 (Q2) and 19.5 (Q3), yielding gaps of 8.6 and 5.8 points, respectively. These patterns suggest that AWeight values around 20–25 may be associated with higher writing proficiency, indicating a potential benchmark for academic vocabulary complexity in argumentative writing. Learners exceeding this range tend to achieve higher scores, while those falling below it are more likely to underperform. This threshold has practical implications: designing instructional strategies and formative feedback centered around boosting learners' academic lexical complexity toward the 20–25 range may contribute to improved outcomes in writing tasks. Additionally, the greater AWeight gap observed in the Q2 task suggests that lexical complexity is more salient in abstract topics such as "hate speech regulation." This underscores the importance of task-specific sensitivity in writing assessment and supports the need for differentiated scoring criteria based on cognitive demands. However, these observational findings would require validation through longitudinal studies and educational interventions to establish causal relationships between vocabulary complexity training and writing improvement. 4.3. Predictive Performance of the Integrated Model of Academic Vocabulary Complexity This section addresses RQ 2 by investigating whether an integrated model combining three academic vocabulary complexity indicators—Academic Diversity (ADiv), Academic Weight (AWeight), and Academic Rate (ARate)—can more effectively explain writing performance than models based on a single indicator. To this end, we employed both multiple linear regression and second-degree polynomial regression. Correlation analyses, feature importance metrics, and residual diagnostics were conducted to evaluate the validity and limitations of the integrated model. 4.3.1 Performance Evaluation of the Multi-Indicator Model In response to RQ 2 − 1, we examined whether the integration of all three indicators yields complementary insights and improves predictive power. As illustrated in Fig. 3 , the linear multi-indicator model produced an R² value of 0.0517, while the second-degree polynomial model slightly improved to 0.0574. Although this represents an 11.0% relative increase, the absolute gain (ΔR² = 0.0057) was marginal and not statistically significant (F-test: p > 0.05). Notably, the integrated model outperformed the AWeight-only model (R² = 0.0497) by just 0.002, indicating limited added value from incorporating ADiv and ARate. This minimal improvement is primarily attributed to severe multicollinearity. As shown in Fig. 3 -(a), AWeight and ARate were highly correlated (r = 0.92), and a substantial correlation was also observed between AWeight and ADiv (r = 0.66). These results suggest that the three indicators largely measure overlapping lexical properties rather than capturing independent or complementary dimensions of academic vocabulary complexity. The instability caused by multicollinearity is evident in the feature importance results in Fig. 3 -(c). ARate appeared to account for 98.2% of the predictive contribution; however, this dominance is likely an artifact of multicollinearity. While ARate showed a positive correlation with writing scores in univariate analysis, its regression coefficient in the multi-indicator model was sharply negative (β = − 13.854). While this might appear to resemble a suppression effect, it is more accurately characterized as computational instability resulting from extreme multicollinearity (r = 0.92). Such computational instability undermines the interpretability of feature importance metrics and further highlights the redundancy among predictors. 4.3.2 Model Diagnostics and Theoretical Implications As shown in Fig. 3 -(d), the residuals from the regression models were approximately normally distributed, with no signs of heteroscedasticity or systematic bias. These results confirm that the basic assumptions of regression were met. Although the explanatory power of the integrated model remained modest (R² = 0.0574), this should be understood in light of the complex, multidimensional nature of writing performance. Writing quality is influenced by numerous factors beyond lexical complexity, including discourse coherence, content relevance, logical structure, and grammatical accuracy. Accordingly, lexical indicators alone are insufficient to account for the full range of evaluative criteria used in writing assessment. Nonetheless, the proposed indicators may serve as supplementary diagnostic tools. While their predictive accuracy is limited, they offer insights into learners’ strategic use of academic vocabulary—particularly in terms of conceptual breadth (ADiv), lexical sophistication (AWeight), and word density (ARate). From an instructional perspective, such insights can be valuable in identifying specific areas for pedagogical intervention. The findings also offer important implications for future modeling. First, the indicators may not fully reflect the dimensions most valued by human raters, such as coherence and argument structure. Second, the strong intercorrelations among indicators point to the need for designing functionally independent measures. Third, future models should expand beyond lexical features to include higher-level textual constructs such as authorial stance, epistemic modality, and rhetorical strategy, which are central to academic writing but difficult to capture with surface-level features alone. In summary, while the integration of academic vocabulary complexity measures did not substantially improve predictive accuracy, the analysis uncovered critical issues related to indicator redundancy and the limitations of purely lexical models. These findings provide a foundation for refining automated writing evaluation systems and underscore the need for more diversified and discourse-sensitive approaches to modeling writing quality. 4.4. Strategic Profiling and Group Differences in Academic Vocabulary Use This section directly addresses RQ 3 by examining distributional patterns across performance groups (RQ3-1) and identifying characteristics of inappropriate complexity cases (RQ3-2). The analysis investigates differences in academic vocabulary complexity strategies between high- and low-performing learners, aiming to (1) identify complexity levels and strategic patterns that influence writing scores, and (2) uncover distinct characteristics of both successful and unsuccessful writers. The analysis consists of three components: (1) comparative analysis of complexity indicators across score groups, (2) empirical examination of inappropriate complexity patterns, and (3) classification of learner strategy types using K-means clustering. 4.4.1. Complexity Differences Across Performance Groups Writers were divided into three groups based on score quartiles: low (n = 628), middle (n = 1,107), and high (n = 694). Statistically significant differences were observed across all three complexity indicators (p < 0.001), as visualized in Fig. 4 -(a), (b), and (c). The average AWeight increased from 18.1 in the low-performing group to 25.5 in the high-performing group, corresponding to a medium effect size (Cohen's d = 0.575). ADiv and ARate also showed significant differences, though with smaller effect sizes (d = 0.435 and d = 0.490, respectively). Compared to the low-performing group, the high-performing group showed a 40.9% increase in AWeight, a 22.4% increase in ADiv, and a 30.6% increase in ARate. However, greater variance within the high-scoring group—particularly in standard deviations—suggests a wide range of strategies even among successful learners. This indicates that strategic adaptability, rather than merely increasing complexity, is critical to high performance. 4.4.2. Analysis of Inappropriate Complexity Patterns To address RQ3-2, we examined cases where learners demonstrated high lexical complexity but achieved low writing scores, operationally defined as learners falling in the top 25% for AWeight but bottom 25% for overall score, in order to capture clear cases of complexity–performance mismatch(n = 106; 4.4% of the total sample). As illustrated in Fig. 4 -(b), these cases are scattered across the high-AWeight range but consistently show low total scores. These learners had a mean AWeight of 42.3 and a mean ARate of 0.092, both substantially above the overall averages (AWeight = 21.7; ARate = 0.056). However, their total scores averaged only 19.2, significantly lower than learners with comparable AWeight but appropriate performance (mean score = 34.8; p < 0.001). Analysis of representative texts from this group revealed several discourse-level characteristics that distinguish inappropriate from effective complexity: Lexical-discourse misalignment Sophisticated academic vocabulary was often used in contexts where simpler, more precise terms would be more appropriate, resulting in stylistic overreach rather than enhanced clarity. Argument-vocabulary disconnect High-level academic terms were frequently employed without clear integration into the logical structure of arguments, creating an impression of surface-level sophistication rather than genuine analytical depth. Register inconsistency Texts exhibited abrupt shifts between highly formal academic vocabulary and more colloquial expressions, disrupting textual coherence and reader comprehension. Semantic precision deficits While vocabulary items were technically accurate, their deployment often lacked semantic precision relative to the specific argumentative context, suggesting strategic vocabulary use without deep conceptual understanding. These findings indicate that inappropriate complexity stems not from lexical inaccuracy but from discourse-level misalignment between vocabulary sophistication and communicative effectiveness. 4.4.3. Learner Typology via Clustering K-means clustering analysis (Fig. 4 -(c), (d)) revealed four distinct strategy types based on their academic vocabulary usage patterns: Cluster 0 (31.4%, n = 762) Low-complexity majority - The largest group, characterized by minimal use of academic vocabulary (AWeight ≈ 17–20) and correspondingly low performance (mean score ≈ 28). This cluster represents learners who have not yet developed substantial academic vocabulary competence. Cluster 1 (28.5%, n = 692) Efficient complexity users - Learners who achieve stable performance (mean score ≈ 31) with moderate complexity (AWeight ≈ 25–30). This group demonstrates balanced strategic use of academic vocabulary, avoiding both under- and over-complexity. Cluster 2 (14.1%, n = 343) High-risk/high-reward strategists - The smallest but highest-performing group (mean score ≈ 35) with extreme complexity (AWeight ≈ 40–45). These learners successfully deploy sophisticated academic vocabulary to achieve superior outcomes, though such high-complexity strategies may entail risks if not accompanied by discourse coherence and precision. Cluster 3 (26.0%, n = 632) Moderate complexity developers - Learners with intermediate complexity levels (AWeight ≈ 22–28) and moderate performance (mean score ≈ 29–30), representing a transitional group with development potential. A notable finding is the absence of a low-complexity/high-performance group, suggesting that academic vocabulary complexity serves as a necessary condition for achieving high scores in the given evaluation context. This threshold effect underscores the gatekeeping function of academic vocabulary in writing assessment. Furthermore, the identification of two successful high-complexity strategies (Clusters 1 and 2) suggests that multiple pathways to effective academic vocabulary use exist. 4.4.4. Strategic Implications and Pedagogical Insights The clustering analysis reveals distinct strategic profiles that have important implications for differentiated instruction: For Cluster 0 learners Priority should be placed on building foundational academic vocabulary repertoires and developing confidence in using moderately complex terms. For Cluster 1 learners Instruction should focus on maintaining strategic balance while gradually expanding vocabulary sophistication without compromising communicative effectiveness. For Cluster 2 learners Advanced learners require guidance in maintaining discourse coherence while employing high-level vocabulary, with emphasis on precision and contextual appropriateness. For Cluster 3 learners This transitional group would benefit from targeted interventions that help them either consolidate moderate complexity strategies or develop toward more sophisticated usage patterns. 4.5. Summary and Implications The analyses presented in this section lead to three key conclusions regarding strategic differences in academic vocabulary use: Threshold Effects and Strategic Variation The complete absence of low-complexity/high-performance learners confirms that academic vocabulary complexity serves as a necessary threshold for writing success. However, the identification of multiple successful strategies (efficient users vs. high-risk strategists) demonstrates that there is no single optimal approach to academic vocabulary deployment. Discourse-Level Determinants of Effectiveness The analysis of inappropriate complexity cases reveals that lexical sophistication alone is insufficient for writing success. Effectiveness depends critically on discourse-level factors including lexical-argument integration, register consistency, and semantic precision. These findings emphasize that vocabulary instruction must address not only word knowledge but also strategic deployment within coherent discourse structures. Differentiated Instructional Implications The four distinct learner profiles suggest that effective academic vocabulary instruction requires differentiated approaches based on learners' current strategic profiles. Rather than uniform complexity enhancement, instruction should be tailored to support learners' progression along identified developmental pathways while addressing specific weaknesses associated with each cluster type. These findings provide empirical foundations for developing adaptive writing instruction and enhancing automated feedback systems, particularly by enabling the identification of learner strategic profiles and tailoring interventions accordingly. 5. General Discussion 5.1. Revisiting the Complexity–Performance Relationship: Implications for Asian Academic Writing Assessment This study empirically examined the influence of academic vocabulary complexity on writing performance, re-evaluating the validity of the hypothesized inverted-U relationship often cited in prior research. The findings revealed a consistently linear upward trend within the observed range of learner data across all complexity indicators (AWeight, ADiv, ARate), with no evidence of performance decline due to excessive complexity. These results contrast with concerns raised in earlier studies (Hinkel, 2003; Norris & Ortega, 2009) that overuse of complex vocabulary may reduce readability and clarity. Within the Korean academic writing context , this linear relationship carries particular significance. In East Asian educational systems, where academic writing serves as a gatekeeping mechanism for university advancement and scholarly participation, the strategic deployment of academic vocabulary represents more than linguistic competency—it embodies cultural and cognitive dimensions of scholarly discourse that are deeply embedded in Confucian educational traditions. The absence of detrimental effects from high complexity usage suggests that Korean L1 writers, unlike L2 learners, possess sufficient linguistic intuition to deploy sophisticated vocabulary without compromising communicative effectiveness. The theoretically predicted peak points (Q2: 86.08, Q3: 65.89) exceeded the observed range, suggesting that the "optimal complexity point" lies beyond learners' current capacities. This finding has profound implications for Korean higher education : rather than cautioning against excessive complexity, pedagogical efforts should focus on expanding learners' capacity to use sophisticated academic vocabulary effectively. The average AWeight of the top 25% of learners remained consistent across tasks (25.8 in Q2, 25.3 in Q3), suggesting the presence of a minimum effective complexity threshold that functions as a necessary condition for high performance in Korean academic discourse. Cross-linguistic implications These findings may extend to other Asian languages that share similar academic writing traditions, particularly those influenced by Classical Chinese scholarly conventions (e.g., Japanese, Vietnamese academic writing). The emphasis on lexical sophistication as a marker of scholarly competence appears to be culturally grounded rather than purely linguistic, suggesting potential applicability of these indicators across East Asian academic contexts. 5.2. Diagnostic Value Over Predictive Accuracy: Reframing Multi-Indicator Assessment for Asian Educational Contexts This study addressed Research Question 2 by examining whether integrating multiple academic vocabulary complexity indicators could provide complementary insights and enhance the interpretability of writing assessment. While the integrated model achieved modest improvements in explanatory power (R² = 0.0574), this finding actually supports theoretical understanding of writing assessment complexity and offers significant advantages for diagnostic applications . Diagnostic Specificity as Theoretical Strength The focused explanatory power (R² = 0.057) demonstrates that academic vocabulary complexity represents a specialized diagnostic dimension rather than a general predictor of writing quality. This specificity is theoretically advantageous because it allows for targeted assessment of lexical sophistication without confounding effects from other writing dimensions such as content relevance or organizational coherence. In language testing contexts, such diagnostic precision is more valuable than broad predictive coverage because it enables instructors to identify specific areas for intervention. The modest R² values confirm that our indicators capture unique variance in writing competence that would otherwise remain invisible in holistic scoring systems. Theoretical Convergence Discovery : The high correlation between AWeight and ARate (r = 0.92) reveals a significant theoretical insight about Korean academic vocabulary acquisition : rather than developing as independent dimensions, vocabulary sophistication and density emerge as convergent aspects of a unified competence . This finding has important implications for understanding L1 academic discourse development. Unlike L2 learners who may develop vocabulary breadth and depth separately, native Korean speakers appear to develop academic vocabulary as an integrated strategic resource . This convergence suggests that successful academic writing in Korean requires not just knowledge of sophisticated terms, but the intuitive ability to deploy them with appropriate density—a finding that could only be discovered through multi-indicator analysis. From a measurement perspective, this convergence validates our theoretical assumption that academic vocabulary complexity represents a coherent construct rather than disparate skills. Cultural specificity and universal applicability The multicollinearity between indicators may reflect language-specific characteristics of Korean academic writing, where dense deployment of sophisticated vocabulary is culturally valued. However, the methodological framework—focusing on meaning-based rather than frequency-based complexity—offers universal applicability for developing culturally sensitive AWE systems across Asian languages. Practical diagnostic advantages Despite limited predictive power, the multi-indicator approach enabled identification of four distinct learner strategic profiles and detection of inappropriate complexity patterns (4.4% of learners). These diagnostic capabilities cannot be achieved through traditional holistic scoring and provide actionable insights for individualized instruction. In resource-constrained Asian educational systems , such targeted diagnostic information offers significant value for optimizing instructional efficiency. AWE system implications Rather than seeking to maximize predictive accuracy, future AWE systems for Asian languages should prioritize diagnostic granularity and cultural sensitivity. The ability to distinguish between "efficient complexity users" and "high-risk strategists" offers pedagogically actionable insights that align with Asian educational values emphasizing strategic competence and contextual appropriateness. 5.3. Strategic Profiling and Cultural Dimensions of Academic Vocabulary Deployment Research Question 3 examined how academic vocabulary complexity indicators can distinguish writing strategies between performance groups and identify characteristics of inappropriate complexity use. The analysis revealed systematic patterns in strategic vocabulary deployment that illuminate both universal principles and culturally specific dimensions of academic writing competence. Threshold Effects and Confucian Educational Values The complete absence of low-complexity/high-performance learners confirms that academic vocabulary complexity serves as a necessary threshold for writing success. This threshold effect aligns with Confucian educational traditions that emphasize mastery of classical forms and sophisticated expression as markers of scholarly achievement. In Korean academic culture, the ability to deploy complex vocabulary appropriately signals not only linguistic competence but also cultural literacy and intellectual maturity. Strategic diversity within cultural constraints The identification of four distinct strategic profiles—foundational vocabulary builders (31.4%), efficient complexity users (28.5%), transitional developers (26.0%), and high-risk strategists (14.1%)—reveals that even within shared cultural expectations, learners develop diverse approaches to academic vocabulary use . This finding challenges assumptions about Asian educational uniformity and supports differentiated instructional approaches that honor individual strategic preferences while maintaining cultural coherence. Discourse-level sophistication beyond lexical knowledge The analysis of inappropriate complexity cases (n = 106; 4.4% of the sample) revealed that lexical sophistication alone does not automatically translate into writing effectiveness. In the Korean context, this finding is particularly significant because it demonstrates that memorization-based vocabulary acquisition—often criticized in Asian educational systems—is insufficient without discourse-level integration skills. The four key discourse-level characteristics that distinguished inappropriate from effective complexity use reflect universal principles of academic writing that transcend cultural boundaries: Lexical-discourse alignment : Sophisticated vocabulary must serve clear communicative functions Argument-vocabulary integration : Academic terms must support rather than obscure logical structure Register consistency : Formal vocabulary must be sustained throughout academic discourse Semantic precision : Vocabulary choices must demonstrate conceptual understanding Implications for Asian language pedagogy These findings suggest that vocabulary instruction in Asian educational contexts should shift from passive memorization toward active discourse integration. The inappropriate complexity phenomenon indicates that traditional approaches emphasizing vocabulary breadth must be supplemented with explicit instruction in strategic deployment and contextual appropriateness. Cross-cultural transferability While the specific threshold values (AWeight 20–25) may be Korean-specific, the strategic profiling framework offers transferable insights for other Asian languages. The emphasis on balancing sophistication with appropriateness reflects universal academic writing principles that can be adapted to diverse linguistic and cultural contexts. 5.4. Implications for Language Testing and Assessment in Asia Advancing culturally responsive assessment This study contributes to the growing movement toward culturally responsive language assessment in Asian contexts. By developing meaning-based complexity indicators that capture Korean-specific academic discourse patterns while maintaining methodological rigor, the research demonstrates how assessment frameworks can honor cultural traditions while meeting international standards. Technology-enhanced assessment potential The diagnostic capabilities demonstrated by these indicators offer significant potential for developing adaptive AWE systems tailored to Asian educational contexts. Unlike Western-developed systems that may not capture Asian academic discourse conventions, culturally grounded indicators can provide more accurate and meaningful feedback for Asian learners. Policy implications for Korean higher education The finding that vocabulary complexity serves as a necessary threshold for writing success supports current emphasis on academic vocabulary instruction in Korean universities. However, the identification of inappropriate complexity patterns suggests that curriculum design should balance vocabulary expansion with discourse integration training. Future directions for Asian language testing research This study establishes foundations for expanding similar investigations across Asian languages, potentially leading to development of Asia-specific AWE systems that capture shared cultural values while accommodating linguistic diversity. The methodological framework—emphasizing meaning over frequency, diagnostic value over predictive accuracy, and strategic profiling over aggregate scoring—offers promising directions for culturally sensitive language assessment research. 6. Conclusion This study developed and validated three meaning-based complexity indices—Academic Diversity (ADiv), Academic Weight (AWeight), and Academic Rate (ARate)—to quantify academic vocabulary usage in Korean undergraduate argumentative writing. Using a large-scale corpus of 2,429 essays, the research confirmed that lexical complexity functions as a necessary but not sufficient condition for writing success, with some learners displaying high sophistication but low performance due to discourse-level weaknesses. Theoretical and Practical Contributions This research makes three key contributions to Asian language testing. First, it establishes a culturally grounded complexity framework that conceptualizes academic vocabulary as 사고도구어 (thinking tool words), reflecting indigenous Korean scholarly discourse traditions rather than applying Western-developed frameworks. Second, it introduces strategic profiling methodology that identifies four distinct learner types—foundational builders, efficient users, transitional developers, and high-risk strategists—demonstrating strategic diversity even within shared cultural expectations. Third, it emphasizes diagnostic precision over predictive breadth , showing that focused indicators (R² = 0.057) enable targeted educational interventions particularly valuable for resource-constrained Asian educational systems. AWeight emerged as the most robust diagnostic indicator , with a benchmark range of 20–25 points identified for effective academic writing in Korean university contexts. The complete absence of low-complexity/high-performance learners confirms academic vocabulary complexity serves a gatekeeping function, validating current educational emphasis on sophisticated vocabulary while highlighting needs for discourse-level integration training. Implications for Assessment and Technology For automated writing evaluation (AWE) , this study provides empirical foundations for developing culturally sensitive systems that serve Asian educational contexts more effectively than Western-developed tools. The identification of "inappropriate complexity" patterns—high sophistication but low performance—demonstrates that effective automated feedback must address discourse-level deployment strategies, not merely vocabulary quantity. The methodological framework offers replicable approaches for developing language-specific complexity indicators across Asian languages sharing similar scholarly traditions, emphasizing meaning-based over frequency-based measures and strategic profiling over aggregate scoring. Future Directions and Significance This research opens avenues for cross-linguistic validation studies across Asian languages, longitudinal investigations of strategic profile development, and genre-specific extensions to other academic writing types. In an era of increasingly Western-centric language assessment , this study demonstrates the vital importance of culturally grounded research that honors local educational traditions while contributing to global academic discourse. The finding that academic vocabulary complexity serves as a necessary threshold, combined with identification of diverse strategic approaches, suggests effective language education must balance universal principles with culturally specific applications. For Korean higher education, this research provides immediately applicable diagnostic tools while validating cultural emphases on lexical sophistication. For Asian language testing broadly, it establishes methodological precedents for assessment frameworks achieving both cultural responsiveness and international scholarly standards—representing a significant contribution to language assessment in increasingly interconnected yet culturally diverse educational contexts. Abbreviations • ADiv Academic Diversity • AWeight Academic Weight • ARate Academic Rate • AV Academic Vocabulary • AWE Automated Writing Evaluation • JSON JavaScript Object Notation • L1 First Language (native language) • L2 Second Language (non-native language) • PCA Principal Component Analysis • RQ Research Question • SHAP SHapley Additive exPlanations • VIF Variance Inflation Factor Declarations Ethics approval and consent to participate This study utilized a publicly available corpus provided by the National Institute of the Korean Language (NIKL), specifically the “Everyone's Corpus (모두의 말뭉치)” section of the 2023 National Corpus for the Development of Korean Writing Proficiency Assessment. All data were originally collected as part of regular curricular writing instruction, with institutional consent procedures in place. No additional ethical approval was required for this secondary analysis of anonymized educational data. Consent for publication Not applicable. Clinical trial number Not applicable. Funding Not applicable. Author Contribution N.K. conceptualized and designed the study, performed the data collection and analysis, developed the academic vocabulary complexity indices, and wrote the main manuscript text. N.K. also prepared all figures and tables. All parts of the manuscript were reviewed and approved by N.K. Acknowledgement The author gratefully acknowledges the National Institute of the Korean Language (NIKL) for providing access to the “Everyone’s Corpus (모두의 말뭉치)” dataset, which served as the foundation for the current analysis. Appreciation is also extended to colleagues who provided insightful comments on earlier drafts of this work. Data Availability The data that support the findings of this study are publicly available from the National Institute of the Korean Language via the “Everyone’s Corpus (모두의 말뭉치)” section at https://corpus.korean.go.krIn addition, the processed data including academic vocabulary complexity indices (ADiv, AWeight, ARate) and the analytical scripts used in this study are available at the following permanent Google Drive repository:https://drive.google.com/drive/folders/15rlRytuDy9xOmq9Ts_fHrAy89P35Qniq?usp=drive_link References Biber, D., & Gray, B. (2010). Challenging stereotypes about academic writing: Complexity, elaboration, explicitness. Journal of English for Academic Purposes , 9 (1), 2–20. https://doi.org/10.1016/j.jeap.2010.01.001 Corson, D. (1997). The learning and use of academic English words. Language Learning , 47 (4), 671–718. https://doi.org/10.1111/0023-8333.00025 Coxhead, A. (2000). A new academic word list. TESOL Quarterly , 34 (2), 213–238. https://doi.org/10.2307/3587951 Kong, N. (2025). Development and validation of academic vocabulary complexity indicators for meaning-centered approaches in automated writing assessment. Korean Semantics , 88 , 263–299. Kyle, K., & Crossley, S. A. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly , 49 (4), 757–786. https://doi.org/10.1002/tesq.194 Laufer, B. (2013). Lexical thresholds for reading comprehension: What they are and how they can be used for teaching purposes. TESOL Quarterly , 47 (4), 867–872. https://doi.org/10.1002/tesq.140 Libben, G. (Ed.). (2007). The representation and processing of compound words . Oxford University Press. Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners' oral narratives. The Modern Language Journal , 96 (2), 190–208. https://doi.org/10.1111/j.1540-4781.2011.01232_1.x Martin, A. V. (1976). Teaching academic vocabulary to foreign graduate students. TESOL Quarterly , 10 (1), 91–97. https://doi.org/10.2307/3585635 Nation, P. (2001). Learning vocabulary in another language . Cambridge University Press. https://doi.org/10.1017/CBO9781139524759 Schmitt, N., & Schmitt, D. (2020). Vocabulary in language teaching (2nd ed.). Cambridge University Press. https://doi.org/10.1017/9781108569057 Shin, M. S. (2004). A study on Korean academic vocabulary education (Doctoral dissertation). Seoul National University, Graduate School, Department of Korean Language Education. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7247858","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":504140952,"identity":"0fd61987-0c26-467b-b205-83588396117d","order_by":0,"name":"Nahyung Kong","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA+klEQVRIiWNgGAWjYNCCA0DM3gBm8oAIxgaitPAcIFmLRAKCj1eLOfvZg595ztjkyUe+MXxc8MdGhoH98APGmXtwa7HsyUuW5rmRVmx4O8fYeGZbGg8DT5oB44ZnuLUYHMgxY+b5cDhx4+wcM2nehsNAv+QwMD44gEfL+TdQLTPPmEnz/PnPw8D/hoCWGyBbbhxOnC/BA9TCdoCHQQJoywa8Wt4YS845k5a4gSet2Ji3LZmHTeKZwcEZeB2WY/jhzTGbxPnthzc+5vljZ8/Pn/zwYQ8eLSDABIo+gwMcBmAeGwMknvACxh9AQr6B/QEhhaNgFIyCUTBCAQD66VLEB7wWmgAAAABJRU5ErkJggg==","orcid":"","institution":"Yonsei University","correspondingAuthor":true,"prefix":"","firstName":"Nahyung","middleName":"","lastName":"Kong","suffix":""}],"badges":[],"createdAt":"2025-07-30 03:08:20","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7247858/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7247858/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":90316308,"identity":"fc16b5d8-5fb2-4eea-b19b-a484dc9b4e99","added_by":"auto","created_at":"2025-09-01 10:20:43","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":116564,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eQ2--Q3 Task Comparison\u003cbr\u003e\n \u003c/strong\u003eDistributions and statistical comparisons of academic vocabulary complexity indicators between Q2 and Q3 writing tasks. Subplots include distribution histograms (a--c), correlation with total score (d), boxplots (e), and effect sizes (f).\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7247858/v1/d7e2b19c0353c4eed2df2bf3.png"},{"id":90316318,"identity":"9196c713-a679-41c3-ab39-efb777b07f07","added_by":"auto","created_at":"2025-09-01 10:20:43","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":268307,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eAWeight Nonlinear Relationship Analysis\u003cbr\u003e\n \u003c/strong\u003eScatterplots and bar charts illustrating the relationship between AWeight (weighted academic vocabulary index) and total score for Q2 and Q3 tasks. Subfigures (a) and (b) display linear and quadratic regression fits, with R² values indicating predictive strength. Subfigures (c) and (d) present mean total scores across AWeight intervals with standard deviation bars, showing a general upward trend for Q2 but a flatter pattern for Q3.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7247858/v1/246147b2f82b9bdc08f6ce56.png"},{"id":90316315,"identity":"8cc401ed-bf72-413b-852a-4912f546f943","added_by":"auto","created_at":"2025-09-01 10:20:43","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":218508,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eMulti-indicator Model Performance\u003cbr\u003e\n \u003c/strong\u003eVisualization of model performance using multiple academic vocabulary complexity indicators. (a) Correlation matrix among ADiv, AWeight, ARate, and total score, showing multicollinearity among predictors. (b) R² scores from regression models using each indicator individually and in combination (linear vs. polynomial). (c) Feature importance analysis indicating ARate as the most influential predictor. (d) Residual plot of the best-performing model, demonstrating acceptable error distribution.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7247858/v1/49189daaaa5a54b69476c8d5.png"},{"id":90316312,"identity":"e54b368b-9041-4392-9dc4-453a15798b61","added_by":"auto","created_at":"2025-09-01 10:20:43","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":298077,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eGroup Differences and Cluster Analysis\u003cbr\u003e\n \u003c/strong\u003eExploratory analysis of AWeight across performance levels and learner profiles. (a) Distribution of AWeight by low, medium, and high scoring groups, with effect size (Cohen's d = 0.516) indicating moderate group differences. (b) Identification of learners with disproportionately high AWeight but low total scores, defined as cases of inappropriate complexity. (c) Results of K-means clustering using total score and AWeight, yielding four distinct learner clusters. (d) Distribution of learners across the four clusters, with respective proportions labeled.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7247858/v1/a6224a5fbdd19a718e8d8448.png"},{"id":109046761,"identity":"08b4cbea-aca0-4bae-82bb-9aab5cedf8df","added_by":"auto","created_at":"2026-05-12 05:43:35","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1155343,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7247858/v1/0aa0afe5-620e-49b4-992e-6bb371ff1783.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Developing Meaning-Based Complexity Indices Based on Cognitive-Functional Connectives and Validating Their Predictive Utility in Automated Writing Evaluation","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eThis study aims to empirically investigate the relationship between academic vocabulary (AV) complexity and writing assessment scores, while also exploring the potential of integrated, multi-indicator predictive models. With the advancement of Automated Writing Evaluation (AWE) systems, there has been growing interest in quantifying linguistic complexity in student writing. However, early studies have primarily relied on surface-level features such as sentence length, word frequency, and syntactic complexity (Kyle \u0026amp; Crossley, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2015\u003c/span\u003e; Lu, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2012\u003c/span\u003e), which have been consistently criticized for their inability to capture writers' cognitive depth and academic reasoning.\u003c/p\u003e\u003cp\u003e\u003cb\u003eIn the Korean educational context, where academic writing serves as a critical gatekeeping mechanism for university advancement and scholarly participation, the strategic deployment of academic vocabulary represents more than linguistic competency\u0026mdash;it embodies cultural and cognitive dimensions of scholarly discourse deeply embedded in East Asian educational traditions.\u003c/b\u003e Unlike Western academic writing that often emphasizes clarity and directness, Korean scholarly discourse values lexical sophistication and conceptual density as markers of intellectual maturity and cultural literacy. This cultural specificity necessitates assessment frameworks that honor indigenous scholarly conventions while maintaining methodological rigor.\u003c/p\u003e\u003cp\u003eTo successfully participate in academic discourse communities, writers must effectively employ field-specific terminology, articulate abstract concepts with precision, and structure logical reasoning within coherent discourse frameworks (Biber \u0026amp; Gray, \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2010\u003c/span\u003e; Corson, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e1997\u003c/span\u003e). These abilities are not adequately represented by simple metrics like word rarity or length, necessitating \u003cb\u003emeaning-based complexity indices\u003c/b\u003e that better reflect cognitive and discourse-level sophistication.\u003c/p\u003e\u003cp\u003eIn particular, \u003cb\u003eacademic vocabulary\u0026mdash;also referred to in the Korean context as 사고도구어 (thinking tool words)\u0026mdash;has become increasingly essential in university-level writing instruction\u003c/b\u003e, where academic literacy is emphasized across disciplines. \u003cb\u003eThe Korean conceptualization of 사고도구어 extends beyond English-based academic word lists to encompass culturally grounded cognitive tools that facilitate abstract reasoning and scholarly argumentation within Confucian intellectual traditions.\u003c/b\u003e In higher education, writing is not only a communicative act but also a cognitive tool for constructing disciplinary knowledge and demonstrating epistemic engagement. Accordingly, students are expected to utilize abstract, metalinguistic expressions to connect, justify, contrast, or condition arguments\u0026mdash;functions that are central to both academic persuasion and conceptual organization.\u003c/p\u003e\u003cp\u003e\u003cb\u003eThis focus on Korean L1 writers addresses a significant gap in Asian language testing research, where most complexity studies have focused on English L2 writing.\u003c/b\u003e By examining native speaker strategic variation in academic vocabulary deployment, this study contributes to understanding how cultural and linguistic factors shape lexical complexity patterns in ways that transcend simple L1/L2 distinctions.\u003c/p\u003e\u003cp\u003eThis study thus focuses on academic vocabulary as a core construct in developing lexical complexity indices that reflect native undergraduate writers' capacity for conceptual development and discourse organization. By operationalizing these meaning-oriented indicators, the study aims to capture cognitively loaded vocabulary patterns that underlie successful academic writing \u003cb\u003ewithin the distinctive cultural and educational context of Korean higher education.\u003c/b\u003e\u003c/p\u003e\u003cp\u003eAccordingly, this study addresses the following research questions:\u003c/p\u003e\u003cp\u003eRQ1. What is the relationship between academic vocabulary complexity and writing assessment scores?\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eRQ1-1. Does the academic vocabulary sophistication index (AWeight) exhibit a non-linear (e.g., inverted U-shaped) relationship with writing scores?\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eRQ1-2. What is the optimal range of AWeight that contributes positively to writing scores, and what effects emerge when this range is exceeded?\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eRQ2. Does a composite model integrating academic vocabulary diversity (ADiv), sophistication (AWeight), and density (ARate) provide complementary insights and improve the interpretability of writing assessment compared to single-indicator models?\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eRQ2-1. What interpretive value or diagnostic potential does the integration of multiple indicators offer beyond predictive accuracy?\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eRQ2-2. How do different indicators contribute to model explanations, and which combinations yield the most informative representations of writing performance?\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eRQ3. How can academic vocabulary complexity indices help distinguish writing strategies between high-scoring and low-scoring groups?\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eRQ3-1. What distributional patterns or strategic tendencies are observed in the use of AV indices across top and bottom scoring groups?\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eRQ3-2. What general patterns or characteristics can be identified in cases where high AV complexity coincides with low writing scores?\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e"},{"header":"2. Literature Review: Academic Vocabulary and Automated Writing Assessment","content":"\u003cp\u003eAcademic vocabulary refers to a set of lexical resources that are essential for higher-order thinking and academic achievement, particularly in formal and scholarly contexts (Martin, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e1976\u003c/span\u003e; Nation, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2001\u003c/span\u003e). Unlike basic conversational vocabulary or domain-specific technical terms, academic vocabulary comprises cognitively functional expressions that facilitate complex meaning-making across disciplines. These lexical items serve to define abstract concepts, organize logical relations, and construct coherent academic discourse.\u003c/p\u003e\u003cp\u003eOne of the key characteristics of academic vocabulary lies in its abstract and metacognitive nature. Rather than reflecting surface-level lexical difficulty, these items function as tools for conceptualization, classification, analysis, comparison, and justification (Corson, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e1997\u003c/span\u003e; Coxhead, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2000\u003c/span\u003e). Accordingly, academic vocabulary plays a crucial role in shaping argumentation and reasoning processes, making it a core component of academic literacy.\u003c/p\u003e\u003cp\u003eFor native speakers, academic vocabulary is not merely a communicative tool but a vehicle for constructing and manipulating abstract knowledge. It has been classified as both \"academic\" and \"cognitive\" vocabulary and is distinguished from field-specific jargon by its cross-disciplinary utility (Schmitt \u0026amp; Schmitt, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). Its role in internalizing and generating academic discourse structures imposes substantial cognitive demands\u0026mdash;even on native speakers with limited academic literacy (Laufer, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2013\u003c/span\u003e). \u003cb\u003eThis study specifically focuses on native Korean undergraduate writers, whose ability to use academic vocabulary reflects their discourse-level academic competence, rather than second-language acquisition.\u003c/b\u003e\u003c/p\u003e\u003cp\u003eIn academic writing, the use of academic vocabulary is closely tied to writing competence. These expressions serve not only as lexical choices but also as strategic devices for constructing claims, signaling reasoning structures, and conveying epistemic stance. Thus, the ability to deploy academic vocabulary effectively is indicative of a writer\u0026rsquo;s capacity for organizing arguments, engaging critically with content, and producing coherent academic discourse\u0026mdash;core dimensions of university-level writing performance.\u003c/p\u003e\u003cp\u003eDespite its importance, academic vocabulary has received limited attention in writing assessment research. Traditional studies have relied on surface-level lexical complexity measures such as the Type-Token Ratio (TTR), Mean Segmental TTR (MSTTR), lexical diversity, and rare word usage rates (Kyle \u0026amp; Crossley, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2015\u003c/span\u003e; Lu, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2012\u003c/span\u003e). While such indicators are useful for gauging linguistic fluency or lexical range, they fail to capture meaning-oriented complexity\u0026mdash;such as discourse function, semantic load, and strategic deployment within a text (Biber \u0026amp; Gray, \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2010\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eIn response to these limitations, recent studies have begun to focus more on the functional roles of vocabulary within discourse rather than on frequency-based metrics. Academic vocabulary has emerged as a central concept in this shift, with empirical research demonstrating its predictive power for reading comprehension, critical thinking, and academic success (Libben et al., 2007). Nonetheless, in the domain of Automated Writing Evaluation (AWE), research quantifying the strategic use of academic vocabulary remains scarce. Most current approaches are limited to detecting basic connectives or calculating lexical diversity in a surface-level manner.\u003c/p\u003e\u003cp\u003eAs AWE systems continue to evolve, there is a growing need to integrate more cognitively and rhetorically grounded scoring features. However, most existing work has emphasized the educational significance of academic vocabulary or its conceptual classification, without sufficiently addressing its operationalization within automated assessment models. This study addresses that gap by proposing meaning-based complexity indices that capture how academic vocabulary is deployed to construct conceptual meaning and discourse structure.\u003c/p\u003e\u003cp\u003eBy incorporating such indices into AWE systems, writing assessment can move beyond structural surface features toward evaluating deeper cognitive and rhetorical dimensions. These indicators can support high-level functions such as identifying high-performing writers, diagnosing ineffective lexical strategies, and providing targeted feedback on discourse construction. This approach holds particular promise for profiling \u003cb\u003enative writers\u0026rsquo; strategic variation in academic vocabulary use\u003c/b\u003e\u0026mdash;a dimension often overlooked in current automated scoring frameworks.\u003c/p\u003e"},{"header":"3. Research Design","content":"\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\u003ch2\u003e3.1. Data and Writing Tasks\u003c/h2\u003e\u003cp\u003eThis study analyzed two argumentative writing tasks from the \u003cem\u003e2023 National Corpus for the Development of Korean Writing Proficiency Assessment\u003c/em\u003e, specifically the \u0026ldquo;Everyone's Corpus(모두의 말뭉치)\u0026rdquo; section compiled by the National Institute of the Korean Language (NIKL). Specifically, we used data from the \u0026ldquo;Everyone's Corpus(모두의 말뭉치)\u0026rdquo; section of the 2023 dataset, which contains a total of 5,000 student-written texts. For this study, 2,429 essays were analyzed: 1,231 responses to Task Q2 (on legal regulations of hate speech) and 1,198 responses to Task Q3 (on restrictions on elderly drivers\u0026rsquo; licenses).\u003c/p\u003e\u003cp\u003eThe corpus consists of argumentative texts written by native speakers of Korean enrolled in undergraduate academic writing courses at university-level institutions in Korea. The participants were all native Korean speakers, primarily first-year students, who were participating in regular curricular writing instruction. The writing tasks were administered as part of official coursework during academic semesters, under standardized test-like conditions (90-minute limit, no access to external resources). These constraints ensured that the essays reflect students\u0026rsquo; authentic academic writing competence under evaluative pressure. This controlled setting ensures that the resulting texts reflect learners\u0026rsquo; authentic academic writing competence within evaluative constraints.\u003c/p\u003e\u003cp\u003e\u003cb\u003eTask Q2\u003c/b\u003e required students to write a logical argument regarding whether hate speech should be legally regulated. The task prompt was as follows:\u003c/p\u003e\u003cp\u003eHate speech refers to verbal or non-verbal acts of public insult, demeaning, contempt, threat, or incitement to discrimination and violence against individuals or groups based on unjustifiable hatred. As hate speech denies the dignity of targeted individuals or groups and has detrimental effects on society, there are increasing calls for its legal regulation. Present your opinion on whether hate speech should be legally regulated, and support your view with logical reasoning.\u003c/p\u003e\u003cp\u003e\u003cb\u003e[Instructions]\u003c/b\u003e Write a complete essay with an introduction, body, and conclusion. Do not include a title. Length: Approximately 1,000 characters (\u0026plusmn;\u0026thinsp;200, including spaces). Time limit: 90 minutes.\u003c/p\u003e\u003cp\u003e\u003cb\u003eTask Q3\u003c/b\u003e asked students to present their opinions on whether elderly drivers should have their licenses restricted. The task prompt was:\u003c/p\u003e\u003cp\u003eIn Korea, anyone aged 18 or older can obtain a driver\u0026rsquo;s license. However, due to recent traffic accidents caused by elderly drivers' slow responses, there are increasing arguments that people should be required to relinquish their licenses at a certain age. As Korean society continues to age, this issue can no longer be ignored. Present your opinion on whether driver\u0026rsquo;s licenses for elderly individuals should be restricted, and support your view with logical reasoning.\u003c/p\u003e\u003cp\u003e\u003cb\u003e[Instructions]\u003c/b\u003e Write a complete essay with an introduction, body, and conclusion. Do not include a title. Length: Approximately 1,000 characters (\u0026plusmn;\u0026thinsp;200, including spaces). Time limit: 90 minutes.\u003c/p\u003e\u003cp\u003eThe two tasks share structural characteristics that make them suitable for comparative analysis: they are both argumentative essays with similar length and time constraints, employ the same evaluation criteria (content, organization, and language use), and explicitly require an introduction\u0026ndash;body\u0026ndash;conclusion format. However, the nature of the topics provides a meaningful contrast for exploring academic vocabulary strategies: Task Q2 involves moral and social value judgments, while Task Q3 requires more policy-oriented and pragmatic reasoning.\u003c/p\u003e\u003cp\u003eAll texts were provided in JSON format and include individual scores for three dimensions\u0026mdash;\u003cb\u003econtent\u003c/b\u003e (eva_score_con), \u003cb\u003eorganization\u003c/b\u003e (eva_score_org), and \u003cb\u003elanguage use\u003c/b\u003e (eva_score_exp)\u0026mdash;as well as a total score. This study used the \u003cb\u003efinal evaluator\u0026rsquo;s\u003c/b\u003e scores as the basis for analysis. Among the evaluated domains, the \u003cb\u003elanguage use\u003c/b\u003e score, which assesses lexical appropriateness, style, and grammatical accuracy, was considered most closely related to the use of academic vocabulary.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e\u003ch2\u003e3.2. Independent Variables: Academic Vocabulary Complexity Indicators\u003c/h2\u003e\u003cp\u003eThis study established three complexity indicators as independent variables to quantitatively measure the usage patterns of \u003cb\u003eacademic vocabulary\u003c/b\u003e, which serves as a core lexical resource reflecting writers\u0026rsquo; cognitive competence in academic discourse. The academic vocabulary list was initially derived from Shin (\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2004\u003c/span\u003e) and was reclassified by the researcher based on discourse functions (see Kong, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2025\u003c/span\u003e). The classification criteria included cognitive functions such as \u003cb\u003ecausation, justification, condition, contrast\u003c/b\u003e, and \u003cb\u003eexemplification\u003c/b\u003e, and each lexical item was graded according to its functional role and level of difficulty.\u003c/p\u003e\u003cp\u003eThe first indicator, \u003cb\u003eAcademic Diversity (ADiv)\u003c/b\u003e, refers to the number of unique academic vocabulary types (i.e., type count) that appear in a text without repetition. ADiv captures the breadth of a writer\u0026rsquo;s academic lexical repertoire, reflecting their ability to strategically activate a range of academic concepts beyond mere lexical variety. It goes beyond surface-level counts to indirectly measure a writer\u0026rsquo;s ability to strategically activate a diverse range of academic concepts. A high ADiv value implies the use of varied thinking strategies and conceptual tools and is positively associated with advanced discourse organization abilities (Biber \u0026amp; Gray, \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2010\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eThe second indicator, \u003cb\u003eAcademic Weight (AWeight)\u003c/b\u003e, is calculated by multiplying each academic term\u0026rsquo;s frequency by its assigned cognitive weight, derived from a four-level difficulty scale within the text. The grading system follows the four-level scheme proposed in Shin (\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2004\u003c/span\u003e), ranging from \u003cb\u003eGrade 1\u003c/b\u003e (basic conceptual terms) to \u003cb\u003eGrade 4\u003c/b\u003e (highly abstract conceptual terms). This indicator aims to reflect not merely the amount but the conceptual complexity and abstractness of the vocabulary used. As such, AWeight serves as a proxy for \u003cb\u003ecognitive depth\u003c/b\u003e, capturing nuances often missed by frequency-based metrics. It also assesses the \u003cb\u003esophistication\u003c/b\u003e and \u003cb\u003eprecision\u003c/b\u003e of vocabulary usage, which underpin academic persuasiveness.\u003c/p\u003e\u003cp\u003eThe third indicator, \u003cb\u003eAcademic Rate (ARate)\u003c/b\u003e, is the proportion (%) of academic vocabulary tokens out of the total tokens in a text. It indicates the density of academic vocabulary usage and can serve as a proxy for the overall \u0026ldquo;academicness\u0026rdquo; of the writing. While ARate helps assess the extent of academic discourse tendencies in learners\u0026rsquo; language, previous studies (e.g., Hinkel, 2003; Norris \u0026amp; Ortega, 2009) caution that excessive density may reduce readability and clarity. Accordingly, ARate can also be useful for detecting \u003cb\u003eoveruse\u003c/b\u003e or \u003cb\u003einappropriate complexity\u003c/b\u003e.\u003c/p\u003e\u003cp\u003eTogether, these three indicators provide \u003cb\u003ecomplementary perspectives\u003c/b\u003e on academic vocabulary usage and offer a multi-dimensional basis for assessing the lexical complexity of learner texts.\u003c/p\u003e\u003cp\u003eAlthough both AWeight and ARate are positively correlated, they capture distinct dimensions of academic vocabulary use. AWeight emphasizes the semantic weight and cognitive sophistication of each vocabulary item, assigning graded difficulty based on its abstractness and functional role. In contrast, ARate measures the proportional density of academic vocabulary relative to the total token count, providing a coarse-grained indicator of overall lexical register. Thus, AWeight represents qualitative complexity\u0026mdash;highlighting the strategic use of cognitively demanding terms\u0026mdash;whereas ARate denotes quantitative saturation of academic vocabulary within a text. Distinguishing these two dimensions is essential for a nuanced interpretation of lexical complexity.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\u003ch2\u003e3.3. Analysis Procedures\u003c/h2\u003e\u003cdiv id=\"Sec7\" class=\"Section3\"\u003e\u003ch2\u003e3.3.1. Variables and Statistical Assumptions\u003c/h2\u003e\u003cp\u003eDependent Variables\u003c/p\u003e\u003cp\u003eThe dependent variables were the total writing score and the expression score (eva_score_exp) provided by the National Institute of Korean Language. The expression score reflects vocabulary appropriateness, grammatical accuracy, and stylistic coherence\u0026mdash;dimensions directly related to academic vocabulary usage. The total score, aggregating content, organization, and expression components, served as the primary indicator of overall writing quality for AWE model evaluation.\u003c/p\u003e\u003cp\u003eStatistical Assumptions and Preprocessing\u003c/p\u003e\u003cp\u003ePrior to analysis, we examined distributional properties and statistical assumptions:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eNormality: Shapiro-Wilk tests and Q-Q plots confirmed approximate normality for dependent variables\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eMulticollinearity: Variance Inflation Factor (VIF) analysis revealed high correlation between AWeight and ARate (r\u0026thinsp;=\u0026thinsp;0.92, VIF\u0026thinsp;\u0026gt;\u0026thinsp;5), necessitating careful interpretation in integrated models\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eOutliers: Values exceeding 3 standard deviations were identified but retained after sensitivity analysis showed minimal impact on results\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eMissing data: Complete case analysis was employed (n\u0026thinsp;=\u0026thinsp;2,429 with no missing values)\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec8\" class=\"Section3\"\u003e\u003ch2\u003e3.3.2. Four-Stage Sequential Analysis Design\u003c/h2\u003e\u003cp\u003eStep 1: Nonlinear Relationship Analysis (RQ1)\u003c/p\u003e\u003cp\u003eObjective: Investigate linear vs. nonlinear relationships between academic vocabulary complexity and writing performance.\u003c/p\u003e\u003cp\u003eMethods:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eModel Comparison: Linear regression vs. quadratic polynomial models for each indicator (ADiv, AWeight, ARate)\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eNonlinearity Testing: Specifically tested inverted U-shaped hypothesis for AWeight using quadratic terms\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eModel Selection: AIC/BIC criteria and nested F-tests for model comparison\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eValidation: 10-fold cross-validation with 80/20 train-test split\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eVisualization: Scatter plots with fitted curves and 95% confidence intervals\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eStatistical Power: Post-hoc power analysis confirmed adequate sample size (n\u0026thinsp;=\u0026thinsp;2,429) for detecting medium effect sizes (f\u0026sup2; \u0026ge; 0.15) with power\u0026thinsp;\u0026gt;\u0026thinsp;0.80.\u003c/p\u003e\u003cp\u003eStep 2: Integrated Model Construction (RQ2-1)\u003c/p\u003e\u003cp\u003eObjective: Evaluate predictive validity and complementary power of combined indicators.\u003c/p\u003e\u003cp\u003eModel Architecture:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eBaseline Linear Model: ADiv\u0026thinsp;+\u0026thinsp;AWeight\u0026thinsp;+\u0026thinsp;ARate as main effects\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eEnhanced Nonlinear Model: Main effects\u0026thinsp;+\u0026thinsp;AWeight\u0026sup2; quadratic term\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eRegularized Model: Ridge regression (α\u0026thinsp;=\u0026thinsp;0.1) to address multicollinearity\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003ePerformance Metrics:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eAccuracy: RMSE, MAE, R\u0026sup2; with 95% confidence intervals\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eValidation: Stratified 10-fold cross-validation with shuffling\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eStability: Bootstrap resampling (n\u0026thinsp;=\u0026thinsp;1,000) for coefficient confidence intervals\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eInterpretability: SHAP (SHapley Additive exPlanations) values for feature importance\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eMulticollinearity Management: Given high VIF values, we employed:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eRidge regression for coefficient stabilization\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003ePrincipal Component Analysis (PCA) as alternative dimensionality reduction\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eVariance decomposition to isolate unique vs. shared variance contributions\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eStep 3: Diagnostic Profiling and Pattern Discovery (RQ2-2)\u003c/p\u003e\u003cp\u003eObjective: Identify learner subtypes through unsupervised clustering of indicator profiles.\u003c/p\u003e\u003cp\u003eClustering Methodology:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eAlgorithm Selection: K-means clustering with elbow method and silhouette analysis for optimal k\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eDistance Metric: Euclidean distance on standardized indicators (z-scores)\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eValidation: Gap statistic and bootstrap clustering stability (Jaccard coefficient\u0026thinsp;\u0026gt;\u0026thinsp;0.75)\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eAlternative Methods: DBSCAN for density-based clustering comparison\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eCluster Interpretation:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eInternal Validation: Within-cluster sum of squares, silhouette coefficients\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eExternal Validation: ANOVA F-tests for cluster differences in writing scores\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eEffect Sizes: Cohen's d for between-cluster comparisons with Bonferroni correction\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003ePedagogical Mapping: Cluster characteristics mapped to instructional implications\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eStep 4: Strategic Group Analysis (RQ3)\u003c/p\u003e\u003cp\u003eObjective: Distinguish writing strategies between performance groups and identify inappropriate complexity patterns.\u003c/p\u003e\u003cp\u003eGroup Definition:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eHigh Performers: Top 25th percentile (n\u0026thinsp;=\u0026thinsp;607, score\u0026thinsp;\u0026ge;\u0026thinsp;36.0)\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eLow Performers: Bottom 25th percentile (n\u0026thinsp;=\u0026thinsp;607, score\u0026thinsp;\u0026le;\u0026thinsp;24.0)\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eInappropriate Complexity: Top 25% AWeight\u0026thinsp;+\u0026thinsp;Bottom 25% total score (n\u0026thinsp;=\u0026thinsp;106, 4.4%)\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eStatistical Comparisons:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eBetween-group Tests: Independent t-tests with Welch's correction for unequal variances\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eEffect Sizes: Cohen's d with 95% confidence intervals, interpreted using Cohen's (1988) benchmarks\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eDistribution Analysis: Kolmogorov-Smirnov tests for distributional differences\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003ePractical Significance: Minimum meaningful difference thresholds based on score ranges\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eQualitative Integration:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eText Selection: Systematic sampling of 20 texts per inappropriate complexity case\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eDiscourse Features: Manual coding for lexical-argument alignment, register consistency, semantic precision\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eInter-rater Reliability: Two independent coders with Cohen's κ\u0026thinsp;\u0026gt;\u0026thinsp;0.80 for qualitative features\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec9\" class=\"Section3\"\u003e\u003ch2\u003e3.3.3. Software and Reproducibility\u003c/h2\u003e\u003cp\u003eAll analyses were conducted in R (v4.3.0) using the packages \u003cem\u003etidyverse\u003c/em\u003e, \u003cem\u003ecaret\u003c/em\u003e, \u003cem\u003ecluster\u003c/em\u003e, \u003cem\u003eSHAP\u003c/em\u003e, and \u003cem\u003eggplot2\u003c/em\u003e. Anonymized learner data and output files (in Excel format) are available via [\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://drive.google.com/drive/folders/15rlRytuDy9xOmq9Ts_fHrAy89P35Qniq?usp=drive_link\u003c/span\u003e\u003cspan address=\"https://drive.google.com/drive/folders/15rlRytuDy9xOmq9Ts_fHrAy89P35Qniq?usp=drive_link\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e] to support transparency and reproducibility. Analysis scripts are available upon request.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e"},{"header":"4. Results","content":"\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\u003ch2\u003e4.1. Descriptive Statistics and Correlation Analysis: Q2\u0026ndash;Q3 Task Comparison\u003c/h2\u003e\u003cp\u003eThis section presents a comparative analysis of academic vocabulary complexity indicators\u0026mdash;Academic Diversity (ADiv), Academic Weight (AWeight), and Academic Rate (ARate)\u0026mdash;across two argumentative writing tasks: Q2 (\u0026ldquo;Legal Regulation of Hate Speech\u0026rdquo;) and Q3 (\u0026ldquo;Restrictions on Elderly Drivers' Licenses\u0026rdquo;). A total of 2,429 texts were analyzed (Q2: 1,231; Q3: 1,198), and the results are visualized in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eFirst, regarding Academic Diversity (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e1\u003c/span\u003e-a), the Q3 task (M\u0026thinsp;=\u0026thinsp;5.205, SD\u0026thinsp;=\u0026thinsp;2.362) yielded a significantly higher average than Q2 (M\u0026thinsp;=\u0026thinsp;4.185, SD\u0026thinsp;=\u0026thinsp;1.973), with a medium effect size (Cohen\u0026rsquo;s d\u0026thinsp;=\u0026thinsp;0.467). The Q3 distribution is clearly shifted to the right, with a wider spread and a maximum value of 15.0 compared to 13.0 in Q2. This suggests that the Q3 prompt elicited a broader conceptual range, likely due to its focus on policy evaluation and real-life implications, which demand diverse academic vocabulary.\u003c/p\u003e\u003cp\u003eSecond, in terms of Academic Weight (Fig.\u0026nbsp;\u0026lt;link rid=\"fig2\"\u0026gt;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e1\u0026lt;/link\u0026gt;\u003c/span\u003e-b and \u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e1\u003c/span\u003e-e), the mean values were relatively similar (Q2: M\u0026thinsp;=\u0026thinsp;21.096; Q3: M\u0026thinsp;=\u0026thinsp;22.301), and the effect size was negligible (d\u0026thinsp;=\u0026thinsp;0.092). Figure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e1\u003c/span\u003e-e shows comparable median AWeight scores across tasks, though Q3 exhibits slightly more variability in the upper quartile and a greater number of outliers. This indicates that while both tasks required cognitively demanding vocabulary, Q3 allowed for greater individual variation in the depth of lexical use.\u003c/p\u003e\u003cp\u003eThird, the Academic Rate distribution (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e1\u003c/span\u003e-c) showed a slightly higher mean for Q2 (M\u0026thinsp;=\u0026thinsp;0.058, SD\u0026thinsp;=\u0026thinsp;0.032) than for Q3 (M\u0026thinsp;=\u0026thinsp;0.054, SD\u0026thinsp;=\u0026thinsp;0.031), with a small effect size (d\u0026thinsp;=\u0026thinsp;0.125). Notably, Q2 exhibits a more peaked distribution around the mode, suggesting more consistent use of academic vocabulary among writers, possibly due to the abstract nature of the hate speech topic.\u003c/p\u003e\u003cp\u003eFourth, correlation analyses (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e1\u003c/span\u003e-d) revealed that AWeight had the strongest relationship with total writing scores in Q2 (r\u0026thinsp;=\u0026thinsp;0.282), while this correlation decreased in Q3 (r\u0026thinsp;=\u0026thinsp;0.162). This finding indicates that conceptually sophisticated vocabulary was more predictive of performance in abstract argumentative contexts. ADiv showed consistent but weaker correlations across tasks (Q2: r\u0026thinsp;=\u0026thinsp;0.174; Q3: r\u0026thinsp;=\u0026thinsp;0.159), while ARate correlated slightly more in Q2 (r\u0026thinsp;=\u0026thinsp;0.194) than Q3 (r\u0026thinsp;=\u0026thinsp;0.167).\u003c/p\u003e\u003cp\u003eFinally, Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e1\u003c/span\u003e-f presents the effect sizes for each indicator across tasks. ADiv demonstrated the clearest task-level differentiation (d\u0026thinsp;=\u0026thinsp;0.467), underscoring its role in capturing topical breadth. In contrast, AWeight and ARate showed minimal effect sizes, suggesting task-invariant patterns in lexical depth and density.\u003c/p\u003e\u003cp\u003eThese findings highlight that academic vocabulary complexity is not uniformly distributed across writing tasks and that task characteristics\u0026mdash;such as topical abstraction and evaluative demand\u0026mdash;can shape the deployment and impact of specific lexical strategies. Such variation provides empirical grounding for task-sensitive modeling in automated writing evaluation and supports the development of writer-specific diagnostic profiles that capture individual patterns of academic vocabulary deployment.\u003c/p\u003e\u003cp\u003eBuilding on these insights, these preliminary findings establish the foundation for addressing our research questions by demonstrating that academic vocabulary complexity patterns are context-dependent, which will inform our subsequent analyses of nonlinear relationships (RQ1), integrated modeling approaches (RQ2), and strategic profiling of learner groups (RQ3).\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e\u003ch2\u003e4.2. Relationship Between Academic Weight (AWeight) and Writing Scores\u003c/h2\u003e\u003cp\u003eAcademic vocabulary complexity is regarded as a key factor that reflects the sophistication of academic reasoning in written discourse. This section analyzes the relationship between one of the complexity indicators\u0026mdash;\u003cb\u003eAcademic Weight (AWeight)\u003c/b\u003e\u0026mdash;and the overall writing scores, aiming to empirically reveal how lexical complexity operates in the actual assessment process. Departing from a purely correlational approach, we employed linear and non-linear regression models as well as interval-based analyses to explore predictive patterns and identify an optimal range of complexity.\u003c/p\u003e\u003cdiv id=\"Sec13\" class=\"Section3\"\u003e\u003ch2\u003e4.2.1. Functional Relationship Between AWeight and Writing Performance\u003c/h2\u003e\u003cp\u003eRQ 1\u0026ndash;1 posits that the relationship between academic vocabulary complexity and writing performance may follow a non-linear curve, potentially exhibiting an inverted-U shape. To examine this hypothesis, both linear (1st-degree) and quadratic (2nd-degree polynomial) regression models were applied, and their predictive power was compared using R\u0026sup2; values alongside theoretical peak points. The results are illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e2\u003c/span\u003e-(a) and (b).\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eScatterplots and bar charts illustrating the relationship between AWeight (weighted academic vocabulary index) and total score for Q2 and Q3 tasks. Subfigures (a) and (b) display linear and quadratic regression fits, with R\u0026sup2; values indicating predictive strength. Subfigures (c) and (d) present mean total scores across AWeight intervals with standard deviation bars, showing a general upward trend for Q2 but a flatter pattern for Q3.\u003c/p\u003e\u003cp\u003eFor the Q2 task, the linear regression model yielded an R\u0026sup2; value of 0.0775, while the quadratic model slightly improved to 0.0785 (ΔR\u0026sup2; = +0.0010). Similarly, in the Q3 task, the linear model had an R\u0026sup2; of 0.0287 and the quadratic model of 0.0308, showing only a marginal difference (ΔR\u0026sup2; = +0.0021) that was not statistically significant (F-test for nested models: Q2: F(1, 1228)\u0026thinsp;=\u0026thinsp;1.30, p\u0026thinsp;\u0026gt;\u0026thinsp;0.1; Q3: F(1, 1195)\u0026thinsp;=\u0026thinsp;2.62, p\u0026thinsp;\u0026gt;\u0026thinsp;0.1). These findings indicate that although AWeight contributes some explanatory power to score prediction, the added value of a non-linear model remains statistically negligible.\u003c/p\u003e\u003cp\u003eAs clearly shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e2\u003c/span\u003e-(a) and (b), the linear and quadratic regression lines nearly overlap across the observed AWeight range, visually confirming the minimal benefit of nonlinear modeling. Furthermore, the theoretical peak values predicted by the quadratic models were AWeight\u0026thinsp;=\u0026thinsp;86.08 for Q2 and AWeight\u0026thinsp;=\u0026thinsp;65.89 for Q3\u0026mdash;both exceeding the observed learner range (Q2: 0\u0026ndash;71; Q3: 0\u0026ndash;75). This suggests potential model overfitting and limits the interpretability of the curve within the current dataset. Within the empirical range, the relationship between AWeight and writing scores appears to be unidirectional and positive, with no observed downturn that would support the inverted-U hypothesis.\u003c/p\u003e\u003cp\u003eTo supplement this interpretation, we conducted an \u003cb\u003einterval-based analysis\u003c/b\u003e (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e2\u003c/span\u003e-(c), (d)). In Q2, learners in the lowest AWeight quartile (average\u0026thinsp;=\u0026thinsp;4.8) achieved a mean score of 26.8, whereas those in the highest quartile (average\u0026thinsp;=\u0026thinsp;41.7) achieved 38.2. In Q3, scores rose from 28.2 (AWeight\u0026thinsp;=\u0026thinsp;5.5) to 33.1 (AWeight\u0026thinsp;=\u0026thinsp;32.1). Crucially, no score decline was observed in the higher AWeight ranges, providing insufficient evidence for the claim that excessive lexical complexity negatively impacts assessment outcomes.\u003c/p\u003e\u003cp\u003eIn summary, AWeight demonstrates a meaningful \u003cb\u003epositive linear relationship\u003c/b\u003e with writing scores. Within the observed learner range, higher AWeight values consistently correspond to improved performance. While the quadratic models suggest a theoretical peak beyond the observed data, such extrapolated interpretations should be treated with caution.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec14\" class=\"Section3\"\u003e\u003ch2\u003e4.2.2. Optimal Range of AWeight and Pedagogical Implications\u003c/h2\u003e\u003cp\u003eRQ 1\u0026ndash;2 sought to determine an efficient range of AWeight that contributes most effectively to writing performance, offering pedagogical insights for instruction and feedback. To this end, we compared the average AWeight of the top and bottom 25% of scorers.\u003c/p\u003e\u003cp\u003eThe analysis revealed that high-scoring learners (top 25%) had an average AWeight of 25.8 in Q2 and 25.3 in Q3, suggesting a \u003cb\u003ecommon complexity level\u003c/b\u003e associated with strong performance, regardless of task type. In contrast, the bottom 25% averaged 17.2 (Q2) and 19.5 (Q3), yielding gaps of 8.6 and 5.8 points, respectively. These patterns suggest that AWeight values around 20\u0026ndash;25 may be associated with higher writing proficiency, indicating a potential benchmark for academic vocabulary complexity in argumentative writing. Learners exceeding this range tend to achieve higher scores, while those falling below it are more likely to underperform.\u003c/p\u003e\u003cp\u003eThis threshold has practical implications: designing instructional strategies and formative feedback centered around boosting learners' academic lexical complexity toward the 20\u0026ndash;25 range may contribute to improved outcomes in writing tasks. Additionally, the greater AWeight gap observed in the Q2 task suggests that lexical complexity is more salient in abstract topics such as \"hate speech regulation.\" This underscores the importance of task-specific sensitivity in writing assessment and supports the need for differentiated scoring criteria based on cognitive demands. However, these observational findings would require validation through longitudinal studies and educational interventions to establish causal relationships between vocabulary complexity training and writing improvement.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Sec15\" class=\"Section2\"\u003e\u003ch2\u003e4.3. Predictive Performance of the Integrated Model of Academic Vocabulary Complexity\u003c/h2\u003e\u003cp\u003eThis section addresses RQ 2 by investigating whether an integrated model combining three academic vocabulary complexity indicators\u0026mdash;Academic Diversity (ADiv), Academic Weight (AWeight), and Academic Rate (ARate)\u0026mdash;can more effectively explain writing performance than models based on a single indicator. To this end, we employed both multiple linear regression and second-degree polynomial regression. Correlation analyses, feature importance metrics, and residual diagnostics were conducted to evaluate the validity and limitations of the integrated model.\u003c/p\u003e\u003cdiv id=\"Sec16\" class=\"Section3\"\u003e\u003ch2\u003e4.3.1 Performance Evaluation of the Multi-Indicator Model\u003c/h2\u003e\u003cp\u003eIn response to RQ 2\u0026thinsp;\u0026minus;\u0026thinsp;1, we examined whether the integration of all three indicators yields complementary insights and improves predictive power. As illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e3\u003c/span\u003e, the linear multi-indicator model produced an R\u0026sup2; value of 0.0517, while the second-degree polynomial model slightly improved to 0.0574. Although this represents an 11.0% relative increase, the absolute gain (ΔR\u0026sup2; = 0.0057) was marginal and not statistically significant (F-test: p\u0026thinsp;\u0026gt;\u0026thinsp;0.05). Notably, the integrated model outperformed the AWeight-only model (R\u0026sup2; = 0.0497) by just 0.002, indicating limited added value from incorporating ADiv and ARate.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eThis minimal improvement is primarily attributed to severe multicollinearity. As shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e3\u003c/span\u003e-(a), AWeight and ARate were highly correlated (r\u0026thinsp;=\u0026thinsp;0.92), and a substantial correlation was also observed between AWeight and ADiv (r\u0026thinsp;=\u0026thinsp;0.66). These results suggest that the three indicators largely measure overlapping lexical properties rather than capturing independent or complementary dimensions of academic vocabulary complexity.\u003c/p\u003e\u003cp\u003eThe instability caused by multicollinearity is evident in the feature importance results in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e3\u003c/span\u003e-(c). ARate appeared to account for 98.2% of the predictive contribution; however, this dominance is likely an artifact of multicollinearity. While ARate showed a positive correlation with writing scores in univariate analysis, its regression coefficient in the multi-indicator model was sharply negative (β = \u0026minus;\u0026thinsp;13.854). While this might appear to resemble a suppression effect, it is more accurately characterized as computational instability resulting from extreme multicollinearity (r\u0026thinsp;=\u0026thinsp;0.92). Such computational instability undermines the interpretability of feature importance metrics and further highlights the redundancy among predictors.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec17\" class=\"Section3\"\u003e\u003ch2\u003e4.3.2 Model Diagnostics and Theoretical Implications\u003c/h2\u003e\u003cp\u003eAs shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e3\u003c/span\u003e-(d), the residuals from the regression models were approximately normally distributed, with no signs of heteroscedasticity or systematic bias. These results confirm that the basic assumptions of regression were met.\u003c/p\u003e\u003cp\u003eAlthough the explanatory power of the integrated model remained modest (R\u0026sup2; = 0.0574), this should be understood in light of the complex, multidimensional nature of writing performance. Writing quality is influenced by numerous factors beyond lexical complexity, including discourse coherence, content relevance, logical structure, and grammatical accuracy. Accordingly, lexical indicators alone are insufficient to account for the full range of evaluative criteria used in writing assessment.\u003c/p\u003e\u003cp\u003eNonetheless, the proposed indicators may serve as supplementary diagnostic tools. While their predictive accuracy is limited, they offer insights into learners\u0026rsquo; strategic use of academic vocabulary\u0026mdash;particularly in terms of conceptual breadth (ADiv), lexical sophistication (AWeight), and word density (ARate). From an instructional perspective, such insights can be valuable in identifying specific areas for pedagogical intervention.\u003c/p\u003e\u003cp\u003eThe findings also offer important implications for future modeling. First, the indicators may not fully reflect the dimensions most valued by human raters, such as coherence and argument structure. Second, the strong intercorrelations among indicators point to the need for designing functionally independent measures. Third, future models should expand beyond lexical features to include higher-level textual constructs such as authorial stance, epistemic modality, and rhetorical strategy, which are central to academic writing but difficult to capture with surface-level features alone.\u003c/p\u003e\u003cp\u003eIn summary, while the integration of academic vocabulary complexity measures did not substantially improve predictive accuracy, the analysis uncovered critical issues related to indicator redundancy and the limitations of purely lexical models. These findings provide a foundation for refining automated writing evaluation systems and underscore the need for more diversified and discourse-sensitive approaches to modeling writing quality.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Sec18\" class=\"Section2\"\u003e\u003ch2\u003e4.4. Strategic Profiling and Group Differences in Academic Vocabulary Use\u003c/h2\u003e\u003cp\u003eThis section directly addresses RQ 3 by examining distributional patterns across performance groups (RQ3-1) and identifying characteristics of inappropriate complexity cases (RQ3-2). The analysis investigates differences in academic vocabulary complexity strategies between high- and low-performing learners, aiming to (1) identify complexity levels and strategic patterns that influence writing scores, and (2) uncover distinct characteristics of both successful and unsuccessful writers. The analysis consists of three components: (1) comparative analysis of complexity indicators across score groups, (2) empirical examination of inappropriate complexity patterns, and (3) classification of learner strategy types using K-means clustering.\u003c/p\u003e\u003cdiv id=\"Sec19\" class=\"Section3\"\u003e\u003ch2\u003e4.4.1. Complexity Differences Across Performance Groups\u003c/h2\u003e\u003cp\u003eWriters were divided into three groups based on score quartiles: low (n\u0026thinsp;=\u0026thinsp;628), middle (n\u0026thinsp;=\u0026thinsp;1,107), and high (n\u0026thinsp;=\u0026thinsp;694). Statistically significant differences were observed across all three complexity indicators (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001), as visualized in Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e4\u003c/span\u003e-(a), (b), and (c).\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eThe average AWeight increased from 18.1 in the low-performing group to 25.5 in the high-performing group, corresponding to a medium effect size (Cohen's d\u0026thinsp;=\u0026thinsp;0.575). ADiv and ARate also showed significant differences, though with smaller effect sizes (d\u0026thinsp;=\u0026thinsp;0.435 and d\u0026thinsp;=\u0026thinsp;0.490, respectively). Compared to the low-performing group, the high-performing group showed a 40.9% increase in AWeight, a 22.4% increase in ADiv, and a 30.6% increase in ARate.\u003c/p\u003e\u003cp\u003eHowever, greater variance within the high-scoring group\u0026mdash;particularly in standard deviations\u0026mdash;suggests a wide range of strategies even among successful learners. This indicates that strategic adaptability, rather than merely increasing complexity, is critical to high performance.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec20\" class=\"Section3\"\u003e\u003ch2\u003e4.4.2. Analysis of Inappropriate Complexity Patterns\u003c/h2\u003e\u003cp\u003eTo address RQ3-2, we examined cases where learners demonstrated high lexical complexity but achieved low writing scores, operationally defined as learners falling in the top 25% for AWeight but bottom 25% for overall score, in order to capture clear cases of complexity\u0026ndash;performance mismatch(n\u0026thinsp;=\u0026thinsp;106; 4.4% of the total sample). As illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e4\u003c/span\u003e-(b), these cases are scattered across the high-AWeight range but consistently show low total scores.\u003c/p\u003e\u003cp\u003eThese learners had a mean AWeight of 42.3 and a mean ARate of 0.092, both substantially above the overall averages (AWeight\u0026thinsp;=\u0026thinsp;21.7; ARate\u0026thinsp;=\u0026thinsp;0.056). However, their total scores averaged only 19.2, significantly lower than learners with comparable AWeight but appropriate performance (mean score\u0026thinsp;=\u0026thinsp;34.8; p\u0026thinsp;\u0026lt;\u0026thinsp;0.001).\u003c/p\u003e\u003cp\u003eAnalysis of representative texts from this group revealed several discourse-level characteristics that distinguish inappropriate from effective complexity:\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eLexical-discourse misalignment\u003c/strong\u003e\u003cp\u003eSophisticated academic vocabulary was often used in contexts where simpler, more precise terms would be more appropriate, resulting in stylistic overreach rather than enhanced clarity.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eArgument-vocabulary disconnect\u003c/strong\u003e\u003cp\u003eHigh-level academic terms were frequently employed without clear integration into the logical structure of arguments, creating an impression of surface-level sophistication rather than genuine analytical depth.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eRegister inconsistency\u003c/strong\u003e\u003cp\u003eTexts exhibited abrupt shifts between highly formal academic vocabulary and more colloquial expressions, disrupting textual coherence and reader comprehension.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eSemantic precision deficits\u003c/strong\u003e\u003cp\u003eWhile vocabulary items were technically accurate, their deployment often lacked semantic precision relative to the specific argumentative context, suggesting strategic vocabulary use without deep conceptual understanding.\u003c/p\u003e\u003c/p\u003e\u003cp\u003eThese findings indicate that inappropriate complexity stems not from lexical inaccuracy but from discourse-level misalignment between vocabulary sophistication and communicative effectiveness.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec21\" class=\"Section3\"\u003e\u003ch2\u003e4.4.3. Learner Typology via Clustering\u003c/h2\u003e\u003cp\u003eK-means clustering analysis (Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e4\u003c/span\u003e-(c), (d)) revealed four distinct strategy types based on their academic vocabulary usage patterns:\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eCluster 0 (31.4%, n\u0026thinsp;=\u0026thinsp;762)\u003c/strong\u003e\u003cp\u003eLow-complexity majority - The largest group, characterized by minimal use of academic vocabulary (AWeight\u0026thinsp;\u0026asymp;\u0026thinsp;17\u0026ndash;20) and correspondingly low performance (mean score\u0026thinsp;\u0026asymp;\u0026thinsp;28). This cluster represents learners who have not yet developed substantial academic vocabulary competence.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eCluster 1 (28.5%, n\u0026thinsp;=\u0026thinsp;692)\u003c/strong\u003e\u003cp\u003eEfficient complexity users - Learners who achieve stable performance (mean score\u0026thinsp;\u0026asymp;\u0026thinsp;31) with moderate complexity (AWeight\u0026thinsp;\u0026asymp;\u0026thinsp;25\u0026ndash;30). This group demonstrates balanced strategic use of academic vocabulary, avoiding both under- and over-complexity.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eCluster 2 (14.1%, n\u0026thinsp;=\u0026thinsp;343)\u003c/strong\u003e\u003cp\u003eHigh-risk/high-reward strategists - The smallest but highest-performing group (mean score\u0026thinsp;\u0026asymp;\u0026thinsp;35) with extreme complexity (AWeight\u0026thinsp;\u0026asymp;\u0026thinsp;40\u0026ndash;45). These learners successfully deploy sophisticated academic vocabulary to achieve superior outcomes, though such high-complexity strategies may entail risks if not accompanied by discourse coherence and precision.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eCluster 3 (26.0%, n\u0026thinsp;=\u0026thinsp;632)\u003c/strong\u003e\u003cp\u003eModerate complexity developers - Learners with intermediate complexity levels (AWeight\u0026thinsp;\u0026asymp;\u0026thinsp;22\u0026ndash;28) and moderate performance (mean score\u0026thinsp;\u0026asymp;\u0026thinsp;29\u0026ndash;30), representing a transitional group with development potential.\u003c/p\u003e\u003c/p\u003e\u003cp\u003eA notable finding is the absence of a low-complexity/high-performance group, suggesting that academic vocabulary complexity serves as a necessary condition for achieving high scores in the given evaluation context. This threshold effect underscores the gatekeeping function of academic vocabulary in writing assessment. Furthermore, the identification of two successful high-complexity strategies (Clusters 1 and 2) suggests that multiple pathways to effective academic vocabulary use exist.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec22\" class=\"Section3\"\u003e\u003ch2\u003e4.4.4. Strategic Implications and Pedagogical Insights\u003c/h2\u003e\u003cp\u003eThe clustering analysis reveals distinct strategic profiles that have important implications for differentiated instruction:\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eFor Cluster 0 learners\u003c/strong\u003e\u003cp\u003ePriority should be placed on building foundational academic vocabulary repertoires and developing confidence in using moderately complex terms.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eFor Cluster 1 learners\u003c/strong\u003e\u003cp\u003eInstruction should focus on maintaining strategic balance while gradually expanding vocabulary sophistication without compromising communicative effectiveness.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eFor Cluster 2 learners\u003c/strong\u003e\u003cp\u003eAdvanced learners require guidance in maintaining discourse coherence while employing high-level vocabulary, with emphasis on precision and contextual appropriateness.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eFor Cluster 3 learners\u003c/strong\u003e\u003cp\u003eThis transitional group would benefit from targeted interventions that help them either consolidate moderate complexity strategies or develop toward more sophisticated usage patterns.\u003c/p\u003e\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv id=\"Sec23\" class=\"Section2\"\u003e\u003ch2\u003e4.5. Summary and Implications\u003c/h2\u003e\u003cp\u003eThe analyses presented in this section lead to three key conclusions regarding strategic differences in academic vocabulary use:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eThreshold Effects and Strategic Variation\u003c/b\u003e\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eThe complete absence of low-complexity/high-performance learners confirms that academic vocabulary complexity serves as a necessary threshold for writing success. However, the identification of multiple successful strategies (efficient users vs. high-risk strategists) demonstrates that there is no single optimal approach to academic vocabulary deployment.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eDiscourse-Level Determinants of Effectiveness\u003c/b\u003e\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eThe analysis of inappropriate complexity cases reveals that lexical sophistication alone is insufficient for writing success. Effectiveness depends critically on discourse-level factors including lexical-argument integration, register consistency, and semantic precision. These findings emphasize that vocabulary instruction must address not only word knowledge but also strategic deployment within coherent discourse structures.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eDifferentiated Instructional Implications\u003c/b\u003e\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eThe four distinct learner profiles suggest that effective academic vocabulary instruction requires differentiated approaches based on learners' current strategic profiles. Rather than uniform complexity enhancement, instruction should be tailored to support learners' progression along identified developmental pathways while addressing specific weaknesses associated with each cluster type.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003eThese findings provide empirical foundations for developing adaptive writing instruction and enhancing automated feedback systems, particularly by enabling the identification of learner strategic profiles and tailoring interventions accordingly.\u003c/p\u003e\u003c/div\u003e"},{"header":"5. General Discussion","content":"\u003cdiv id=\"Sec25\" class=\"Section2\"\u003e\u003ch2\u003e5.1. Revisiting the Complexity\u0026ndash;Performance Relationship: Implications for Asian Academic Writing Assessment\u003c/h2\u003e\u003cp\u003eThis study empirically examined the influence of academic vocabulary complexity on writing performance, re-evaluating the validity of the hypothesized inverted-U relationship often cited in prior research. The findings revealed a consistently linear upward trend \u003cb\u003ewithin the observed range of learner data\u003c/b\u003e across all complexity indicators (AWeight, ADiv, ARate), with no evidence of performance decline due to excessive complexity. These results contrast with concerns raised in earlier studies (Hinkel, 2003; Norris \u0026amp; Ortega, 2009) that overuse of complex vocabulary may reduce readability and clarity.\u003c/p\u003e\u003cp\u003e\u003cb\u003eWithin the Korean academic writing context\u003c/b\u003e, this linear relationship carries particular significance. In East Asian educational systems, where academic writing serves as a gatekeeping mechanism for university advancement and scholarly participation, the strategic deployment of academic vocabulary represents more than linguistic competency\u0026mdash;it embodies cultural and cognitive dimensions of scholarly discourse that are deeply embedded in Confucian educational traditions. The absence of detrimental effects from high complexity usage suggests that Korean L1 writers, unlike L2 learners, possess sufficient linguistic intuition to deploy sophisticated vocabulary without compromising communicative effectiveness.\u003c/p\u003e\u003cp\u003eThe theoretically predicted peak points (Q2: 86.08, Q3: 65.89) exceeded the observed range, suggesting that the \"optimal complexity point\" lies beyond learners' current capacities. \u003cb\u003eThis finding has profound implications for Korean higher education\u003c/b\u003e: rather than cautioning against excessive complexity, pedagogical efforts should focus on expanding learners' capacity to use sophisticated academic vocabulary effectively. The average AWeight of the top 25% of learners remained consistent across tasks (25.8 in Q2, 25.3 in Q3), suggesting the presence of a \u003cb\u003eminimum effective complexity threshold\u003c/b\u003e that functions as a necessary condition for high performance in Korean academic discourse.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eCross-linguistic implications\u003c/strong\u003e\u003cp\u003eThese findings may extend to other Asian languages that share similar academic writing traditions, particularly those influenced by Classical Chinese scholarly conventions (e.g., Japanese, Vietnamese academic writing). The emphasis on lexical sophistication as a marker of scholarly competence appears to be culturally grounded rather than purely linguistic, suggesting potential applicability of these indicators across East Asian academic contexts.\u003c/p\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec26\" class=\"Section2\"\u003e\u003ch2\u003e5.2. Diagnostic Value Over Predictive Accuracy: Reframing Multi-Indicator Assessment for Asian Educational Contexts\u003c/h2\u003e\u003cp\u003eThis study addressed Research Question 2 by examining whether integrating multiple academic vocabulary complexity indicators could provide complementary insights and enhance the interpretability of writing assessment. While the integrated model achieved modest improvements in explanatory power (R\u0026sup2; = 0.0574), \u003cb\u003ethis finding actually supports theoretical understanding of writing assessment complexity and offers significant advantages for diagnostic applications\u003c/b\u003e.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eDiagnostic Specificity as Theoretical Strength\u003c/strong\u003e\u003cp\u003eThe focused explanatory power (R\u0026sup2; = 0.057) demonstrates that academic vocabulary complexity represents a \u003cb\u003especialized diagnostic dimension\u003c/b\u003e rather than a general predictor of writing quality. This specificity is theoretically advantageous because it allows for \u003cb\u003etargeted assessment of lexical sophistication\u003c/b\u003e without confounding effects from other writing dimensions such as content relevance or organizational coherence.\u003c/p\u003e\u003c/p\u003e\u003cp\u003eIn language testing contexts, such \u003cb\u003ediagnostic precision\u003c/b\u003e is more valuable than broad predictive coverage because it enables instructors to identify specific areas for intervention. The modest R\u0026sup2; values confirm that our indicators capture \u003cb\u003eunique variance\u003c/b\u003e in writing competence that would otherwise remain invisible in holistic scoring systems.\u003c/p\u003e\u003cp\u003e\u003cb\u003eTheoretical Convergence Discovery\u003c/b\u003e: The high correlation between AWeight and ARate (r\u0026thinsp;=\u0026thinsp;0.92) reveals a significant \u003cb\u003etheoretical insight about Korean academic vocabulary acquisition\u003c/b\u003e: rather than developing as independent dimensions, vocabulary sophistication and density emerge as \u003cb\u003econvergent aspects of a unified competence\u003c/b\u003e.\u003c/p\u003e\u003cp\u003eThis finding has important implications for understanding L1 academic discourse development. Unlike L2 learners who may develop vocabulary breadth and depth separately, \u003cb\u003enative Korean speakers appear to develop academic vocabulary as an integrated strategic resource\u003c/b\u003e. This convergence suggests that successful academic writing in Korean requires not just knowledge of sophisticated terms, but the intuitive ability to deploy them with appropriate density\u0026mdash;a finding that could only be discovered through multi-indicator analysis.\u003c/p\u003e\u003cp\u003eFrom a measurement perspective, this convergence validates our theoretical assumption that academic vocabulary complexity represents a \u003cb\u003ecoherent construct\u003c/b\u003e rather than disparate skills.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eCultural specificity and universal applicability\u003c/strong\u003e\u003cp\u003eThe multicollinearity between indicators may reflect language-specific characteristics of Korean academic writing, where dense deployment of sophisticated vocabulary is culturally valued. However, the methodological framework\u0026mdash;focusing on meaning-based rather than frequency-based complexity\u0026mdash;offers universal applicability for developing culturally sensitive AWE systems across Asian languages.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003ePractical diagnostic advantages\u003c/strong\u003e\u003cp\u003eDespite limited predictive power, the multi-indicator approach enabled identification of four distinct learner strategic profiles and detection of inappropriate complexity patterns (4.4% of learners). These diagnostic capabilities cannot be achieved through traditional holistic scoring and provide actionable insights for individualized instruction. \u003cb\u003eIn resource-constrained Asian educational systems\u003c/b\u003e, such targeted diagnostic information offers significant value for optimizing instructional efficiency.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eAWE system implications\u003c/strong\u003e\u003cp\u003eRather than seeking to maximize predictive accuracy, future AWE systems for Asian languages should prioritize diagnostic granularity and cultural sensitivity. The ability to distinguish between \"efficient complexity users\" and \"high-risk strategists\" offers pedagogically actionable insights that align with Asian educational values emphasizing strategic competence and contextual appropriateness.\u003c/p\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec27\" class=\"Section2\"\u003e\u003ch2\u003e5.3. Strategic Profiling and Cultural Dimensions of Academic Vocabulary Deployment\u003c/h2\u003e\u003cp\u003eResearch Question 3 examined how academic vocabulary complexity indicators can distinguish writing strategies between performance groups and identify characteristics of inappropriate complexity use. The analysis revealed systematic patterns in strategic vocabulary deployment that illuminate both universal principles and culturally specific dimensions of academic writing competence.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eThreshold Effects and Confucian Educational Values\u003c/strong\u003e\u003cp\u003eThe complete absence of low-complexity/high-performance learners confirms that academic vocabulary complexity serves as a necessary threshold for writing success. \u003cb\u003eThis threshold effect aligns with Confucian educational traditions\u003c/b\u003e that emphasize mastery of classical forms and sophisticated expression as markers of scholarly achievement. In Korean academic culture, the ability to deploy complex vocabulary appropriately signals not only linguistic competence but also cultural literacy and intellectual maturity.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eStrategic diversity within cultural constraints\u003c/strong\u003e\u003cp\u003eThe identification of four distinct strategic profiles\u0026mdash;foundational vocabulary builders (31.4%), efficient complexity users (28.5%), transitional developers (26.0%), and high-risk strategists (14.1%)\u0026mdash;reveals that \u003cb\u003eeven within shared cultural expectations, learners develop diverse approaches to academic vocabulary use\u003c/b\u003e. This finding challenges assumptions about Asian educational uniformity and supports differentiated instructional approaches that honor individual strategic preferences while maintaining cultural coherence.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eDiscourse-level sophistication beyond lexical knowledge\u003c/strong\u003e\u003cp\u003eThe analysis of inappropriate complexity cases (n\u0026thinsp;=\u0026thinsp;106; 4.4% of the sample) revealed that lexical sophistication alone does not automatically translate into writing effectiveness. \u003cb\u003eIn the Korean context, this finding is particularly significant\u003c/b\u003e because it demonstrates that memorization-based vocabulary acquisition\u0026mdash;often criticized in Asian educational systems\u0026mdash;is insufficient without discourse-level integration skills.\u003c/p\u003e\u003c/p\u003e\u003cp\u003eThe four key discourse-level characteristics that distinguished inappropriate from effective complexity use reflect universal principles of academic writing that transcend cultural boundaries:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eLexical-discourse alignment\u003c/b\u003e: Sophisticated vocabulary must serve clear communicative functions\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eArgument-vocabulary integration\u003c/b\u003e: Academic terms must support rather than obscure logical structure\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eRegister consistency\u003c/b\u003e: Formal vocabulary must be sustained throughout academic discourse\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e\u003cb\u003eSemantic precision\u003c/b\u003e: Vocabulary choices must demonstrate conceptual understanding\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eImplications for Asian language pedagogy\u003c/strong\u003e\u003cp\u003eThese findings suggest that vocabulary instruction in Asian educational contexts should shift from passive memorization toward active discourse integration. The inappropriate complexity phenomenon indicates that traditional approaches emphasizing vocabulary breadth must be supplemented with explicit instruction in strategic deployment and contextual appropriateness.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eCross-cultural transferability\u003c/strong\u003e\u003cp\u003eWhile the specific threshold values (AWeight 20\u0026ndash;25) may be Korean-specific, the strategic profiling framework offers transferable insights for other Asian languages. The emphasis on balancing sophistication with appropriateness reflects universal academic writing principles that can be adapted to diverse linguistic and cultural contexts.\u003c/p\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec28\" class=\"Section2\"\u003e\u003ch2\u003e5.4. Implications for Language Testing and Assessment in Asia\u003c/h2\u003e\u003cp\u003e\u003cstrong\u003eAdvancing culturally responsive assessment\u003c/strong\u003e\u003cp\u003eThis study contributes to the growing movement toward culturally responsive language assessment in Asian contexts. By developing meaning-based complexity indicators that capture Korean-specific academic discourse patterns while maintaining methodological rigor, the research demonstrates how assessment frameworks can honor cultural traditions while meeting international standards.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eTechnology-enhanced assessment potential\u003c/strong\u003e\u003cp\u003eThe diagnostic capabilities demonstrated by these indicators offer significant potential for developing adaptive AWE systems tailored to Asian educational contexts. Unlike Western-developed systems that may not capture Asian academic discourse conventions, culturally grounded indicators can provide more accurate and meaningful feedback for Asian learners.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003ePolicy implications for Korean higher education\u003c/strong\u003e\u003cp\u003eThe finding that vocabulary complexity serves as a necessary threshold for writing success supports current emphasis on academic vocabulary instruction in Korean universities. However, the identification of inappropriate complexity patterns suggests that curriculum design should balance vocabulary expansion with discourse integration training.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eFuture directions for Asian language testing research\u003c/strong\u003e\u003cp\u003eThis study establishes foundations for expanding similar investigations across Asian languages, potentially leading to development of Asia-specific AWE systems that capture shared cultural values while accommodating linguistic diversity. The methodological framework\u0026mdash;emphasizing meaning over frequency, diagnostic value over predictive accuracy, and strategic profiling over aggregate scoring\u0026mdash;offers promising directions for culturally sensitive language assessment research.\u003c/p\u003e\u003c/p\u003e\u003c/div\u003e"},{"header":"6. Conclusion","content":"\u003cp\u003eThis study developed and validated three meaning-based complexity indices\u0026mdash;Academic Diversity (ADiv), Academic Weight (AWeight), and Academic Rate (ARate)\u0026mdash;to quantify academic vocabulary usage in Korean undergraduate argumentative writing. Using a large-scale corpus of 2,429 essays, the research confirmed that lexical complexity functions as a necessary but not sufficient condition for writing success, with some learners displaying high sophistication but low performance due to discourse-level weaknesses.\u003c/p\u003e\u003cp\u003e\u003cb\u003eTheoretical and Practical Contributions\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThis research makes three key contributions to Asian language testing. First, it establishes a \u003cb\u003eculturally grounded complexity framework\u003c/b\u003e that conceptualizes academic vocabulary as 사고도구어 (thinking tool words), reflecting indigenous Korean scholarly discourse traditions rather than applying Western-developed frameworks. Second, it introduces \u003cb\u003estrategic profiling methodology\u003c/b\u003e that identifies four distinct learner types\u0026mdash;foundational builders, efficient users, transitional developers, and high-risk strategists\u0026mdash;demonstrating strategic diversity even within shared cultural expectations. Third, it emphasizes \u003cb\u003ediagnostic precision over predictive breadth\u003c/b\u003e, showing that focused indicators (R\u0026sup2; = 0.057) enable targeted educational interventions particularly valuable for resource-constrained Asian educational systems.\u003c/p\u003e\u003cp\u003e\u003cb\u003eAWeight emerged as the most robust diagnostic indicator\u003c/b\u003e, with a benchmark range of 20\u0026ndash;25 points identified for effective academic writing in Korean university contexts. The complete absence of low-complexity/high-performance learners confirms academic vocabulary complexity serves a gatekeeping function, validating current educational emphasis on sophisticated vocabulary while highlighting needs for discourse-level integration training.\u003c/p\u003e\u003cp\u003e\u003cb\u003eImplications for Assessment and Technology\u003c/b\u003e\u003c/p\u003e\u003cp\u003eFor \u003cb\u003eautomated writing evaluation (AWE)\u003c/b\u003e, this study provides empirical foundations for developing culturally sensitive systems that serve Asian educational contexts more effectively than Western-developed tools. The identification of \"inappropriate complexity\" patterns\u0026mdash;high sophistication but low performance\u0026mdash;demonstrates that effective automated feedback must address discourse-level deployment strategies, not merely vocabulary quantity.\u003c/p\u003e\u003cp\u003eThe methodological framework offers \u003cb\u003ereplicable approaches\u003c/b\u003e for developing language-specific complexity indicators across Asian languages sharing similar scholarly traditions, emphasizing meaning-based over frequency-based measures and strategic profiling over aggregate scoring.\u003c/p\u003e\u003cp\u003e\u003cb\u003eFuture Directions and Significance\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThis research opens avenues for cross-linguistic validation studies across Asian languages, longitudinal investigations of strategic profile development, and genre-specific extensions to other academic writing types.\u003c/p\u003e\u003cp\u003e\u003cb\u003eIn an era of increasingly Western-centric language assessment\u003c/b\u003e, this study demonstrates the vital importance of culturally grounded research that honors local educational traditions while contributing to global academic discourse. The finding that academic vocabulary complexity serves as a necessary threshold, combined with identification of diverse strategic approaches, suggests effective language education must balance universal principles with culturally specific applications.\u003c/p\u003e\u003cp\u003eFor Korean higher education, this research provides immediately applicable diagnostic tools while validating cultural emphases on lexical sophistication. For Asian language testing broadly, it establishes methodological precedents for assessment frameworks achieving both cultural responsiveness and international scholarly standards\u0026mdash;representing a significant contribution to language assessment in increasingly interconnected yet culturally diverse educational contexts.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eADiv\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eAcademic Diversity\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eAWeight\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eAcademic Weight\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eARate\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eAcademic Rate\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eAV\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eAcademic Vocabulary\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eAWE\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eAutomated Writing Evaluation\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eJSON\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eJavaScript Object Notation\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eL1\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eFirst Language (native language)\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eL2\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eSecond Language (non-native language)\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003ePCA\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003ePrincipal Component Analysis\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eRQ\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eResearch Question\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eSHAP\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eSHapley Additive exPlanations\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003e\u0026bull; \u003cb\u003eVIF\u003c/b\u003e\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eVariance Inflation Factor\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003cp\u003eThis study utilized a publicly available corpus provided by the National Institute of the Korean Language (NIKL), specifically the \u003cem\u003e\u0026ldquo;Everyone's Corpus (모두의 말뭉치)\u0026rdquo;\u003c/em\u003e section of the 2023 National Corpus for the Development of Korean Writing Proficiency Assessment. All data were originally collected as part of regular curricular writing instruction, with institutional consent procedures in place. No additional ethical approval was required for this secondary analysis of anonymized educational data.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003cp\u003eNot applicable.\u003c/p\u003e\u003c/p\u003e\u003cp\u003e\u003ch2\u003eClinical trial number\u003c/h2\u003e\u003cp\u003eNot applicable.\u003c/p\u003e\u003c/p\u003e\u003ch2\u003eFunding\u003c/h2\u003e\u003cp\u003eNot applicable.\u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eN.K. conceptualized and designed the study, performed the data collection and analysis, developed the academic vocabulary complexity indices, and wrote the main manuscript text. N.K. also prepared all figures and tables. All parts of the manuscript were reviewed and approved by N.K.\u003c/p\u003e\u003ch2\u003eAcknowledgement\u003c/h2\u003e\u003cp\u003eThe author gratefully acknowledges the National Institute of the Korean Language (NIKL) for providing access to the \u0026ldquo;Everyone\u0026rsquo;s Corpus (모두의 말뭉치)\u0026rdquo; dataset, which served as the foundation for the current analysis. Appreciation is also extended to colleagues who provided insightful comments on earlier drafts of this work.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe data that support the findings of this study are publicly available from the National Institute of the Korean Language via the \u0026ldquo;Everyone\u0026rsquo;s Corpus (모두의 말뭉치)\u0026rdquo; section at https://corpus.korean.go.krIn addition, the processed data including academic vocabulary complexity indices (ADiv, AWeight, ARate) and the analytical scripts used in this study are available at the following permanent Google Drive repository:https://drive.google.com/drive/folders/15rlRytuDy9xOmq9Ts_fHrAy89P35Qniq?usp=drive_link\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eBiber, D., \u0026amp; Gray, B. (2010). Challenging stereotypes about academic writing: Complexity, elaboration, explicitness. \u003cem\u003eJournal of English for Academic Purposes\u003c/em\u003e, \u003cem\u003e9\u003c/em\u003e(1), 2\u0026ndash;20. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.jeap.2010.01.001\u003c/span\u003e\u003cspan address=\"10.1016/j.jeap.2010.01.001\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCorson, D. (1997). The learning and use of academic English words. \u003cem\u003eLanguage Learning\u003c/em\u003e, \u003cem\u003e47\u003c/em\u003e(4), 671\u0026ndash;718. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1111/0023-8333.00025\u003c/span\u003e\u003cspan address=\"10.1111/0023-8333.00025\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCoxhead, A. (2000). A new academic word list. \u003cem\u003eTESOL Quarterly\u003c/em\u003e, \u003cem\u003e34\u003c/em\u003e(2), 213\u0026ndash;238. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.2307/3587951\u003c/span\u003e\u003cspan address=\"10.2307/3587951\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKong, N. (2025). Development and validation of academic vocabulary complexity indicators for meaning-centered approaches in automated writing assessment. \u003cem\u003eKorean Semantics\u003c/em\u003e, \u003cem\u003e88\u003c/em\u003e, 263\u0026ndash;299.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKyle, K., \u0026amp; Crossley, S. A. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. \u003cem\u003eTESOL Quarterly\u003c/em\u003e, \u003cem\u003e49\u003c/em\u003e(4), 757\u0026ndash;786. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/tesq.194\u003c/span\u003e\u003cspan address=\"10.1002/tesq.194\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLaufer, B. (2013). Lexical thresholds for reading comprehension: What they are and how they can be used for teaching purposes. \u003cem\u003eTESOL Quarterly\u003c/em\u003e, \u003cem\u003e47\u003c/em\u003e(4), 867\u0026ndash;872. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/tesq.140\u003c/span\u003e\u003cspan address=\"10.1002/tesq.140\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLibben, G. (Ed.). (2007). \u003cem\u003eThe representation and processing of compound words\u003c/em\u003e. Oxford University Press.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLu, X. (2012). The relationship of lexical richness to the quality of ESL learners' oral narratives. \u003cem\u003eThe Modern Language Journal\u003c/em\u003e, \u003cem\u003e96\u003c/em\u003e(2), 190\u0026ndash;208. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1111/j.1540-4781.2011.01232_1.x\u003c/span\u003e\u003cspan address=\"10.1111/j.1540-4781.2011.01232_1.x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMartin, A. V. (1976). Teaching academic vocabulary to foreign graduate students. \u003cem\u003eTESOL Quarterly\u003c/em\u003e, \u003cem\u003e10\u003c/em\u003e(1), 91\u0026ndash;97. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.2307/3585635\u003c/span\u003e\u003cspan address=\"10.2307/3585635\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eNation, P. (2001). \u003cem\u003eLearning vocabulary in another language\u003c/em\u003e. Cambridge University Press. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1017/CBO9781139524759\u003c/span\u003e\u003cspan address=\"10.1017/CBO9781139524759\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSchmitt, N., \u0026amp; Schmitt, D. (2020). \u003cem\u003eVocabulary in language teaching\u003c/em\u003e (2nd ed.). Cambridge University Press. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1017/9781108569057\u003c/span\u003e\u003cspan address=\"10.1017/9781108569057\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eShin, M. S. (2004). \u003cem\u003eA study on Korean academic vocabulary education\u003c/em\u003e (Doctoral dissertation). Seoul National University, Graduate School, Department of Korean Language Education.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"academic writing, argumentative writing, academic vocabulary, lexical complexity, native Korean writers, automated writing evaluation, argumentative discourse","lastPublishedDoi":"10.21203/rs.3.rs-7247858/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7247858/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis study develops and validates three meaning-based complexity indices\u0026mdash;\u003cb\u003eAcademic Diversity (ADiv), Academic Weight (AWeight), and Academic Rate (ARate)\u003c/b\u003e\u0026mdash;to quantify academic vocabulary usage in native Korean undergraduate students\u0026rsquo; argumentative writing and assess their predictive value in writing evaluation. Drawing on a large-scale national writing assessment corpus comprising 2,429 essays from two academic prompts, the study investigates how these indices reflect cognitive depth and discourse competence in L1 academic writing.\u003c/p\u003e\u003cp\u003eRegression analyses reveal that \u003cb\u003eAWeight\u003c/b\u003e, which captures the semantic sophistication of academic terms based on graded difficulty, exhibits a consistently positive relationship with writing scores. Contrary to prior assumptions of an inverted-U shape, no detrimental effects of \u0026ldquo;excessive complexity\u0026rdquo; were observed within the actual learner range. AWeight also emerged as the most effective diagnostic indicator for distinguishing high- and low-scoring writers, with a potential benchmark range identified between 20\u0026ndash;25 points.\u003c/p\u003e\u003cp\u003eWhile integrated models incorporating all three indices yielded only marginal improvements in predictive power (R\u0026sup2; = 0.0574), \u003cb\u003eclustering analyses revealed distinct lexical strategy types\u003c/b\u003e, highlighting variation in how native speakers deploy academic vocabulary. Notably, a subgroup exhibiting \u0026ldquo;inappropriate complexity\u0026rdquo;\u0026mdash;high lexical sophistication but low scores\u0026mdash;suggests that lexical complexity alone is insufficient without discourse-level alignment.\u003c/p\u003e\u003cp\u003eThese findings underscore the \u003cb\u003egatekeeping function\u003c/b\u003e of academic vocabulary complexity in automated writing evaluation (AWE) systems and call for multi-dimensional assessment frameworks that integrate \u003cb\u003esemantic depth, discourse appropriateness, and strategic deployment\u003c/b\u003e. The study contributes theoretically by reframing lexical complexity as a \u003cb\u003econtext-sensitive, functional construct\u003c/b\u003e, and practically by proposing pedagogically interpretable indices for AWE and learner diagnostics in native academic writing contexts.\u003c/p\u003e","manuscriptTitle":"Developing Meaning-Based Complexity Indices Based on Cognitive-Functional Connectives and Validating Their Predictive Utility in Automated Writing Evaluation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-01 10:20:38","doi":"10.21203/rs.3.rs-7247858/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ef2f48c3-27f6-47eb-8687-b6a80b0d55a4","owner":[],"postedDate":"September 1st, 2025","published":true,"recentEditorialEvents":[{"type":"decision","content":"Withdrawn","date":"2026-05-12T05:32:23+00:00","index":"","fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-12T05:43:05+00:00","versionOfRecord":[],"versionCreatedAt":"2025-09-01 10:20:38","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7247858","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7247858","identity":"rs-7247858","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}