Understanding the Impact of Emotional Engagement on Learning Outcomes in Online Education: An Automated Analysis Approach

Understanding the Impact of Emotional Engagement on Learning Outcomes in Online Education: An Automated Analysis Approach | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Understanding the Impact of Emotional Engagement on Learning Outcomes in Online Education: An Automated Analysis Approach Guanyu Chen, Guangxin Han, Juan Niu, Juhou He This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7581467/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 07 Jan, 2026 Read the published version in Scientific Reports → Version 1 posted 10 You are reading this latest preprint version Abstract Online education offers flexibility but often suffers from reduced learner engagement. This study developed an automated method to detect emotional engagement using an optimized Vision Transformer model with transfer learning. Facial data from 40 undergraduates produced a dataset of 71,185 labeled images across three engagement levels. The proposed model achieved 93.8% classification accuracy, surpassing conventional machine learning and deep learning baselines. Analysis showed engagement typically declined after six minutes of learning, with a modest rebound near session end. Pearson correlation revealed a significant positive relationship between engagement and learning outcomes, indicating that emotionally engaged learners achieved higher academic performance. These results demonstrate the feasibility of deep learning–based approaches for scalable monitoring of learner engagement and highlight the central role of emotional states in shaping online learning effectiveness. The findings provide practical insights for designing adaptive interventions to sustain attention and optimize digital learning environments. Physical sciences/Mathematics and computing Biological sciences/Psychology Social science/Psychology Emotional Engagement Online Learning Artificial Intelligence in Education Educational Data Mining Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Introduction With the rapid progress and technological advancements in education, internet services have gained widespread adoption and implementation across major universities, as well as primary and secondary schools. Consequently, online education has undergone substantial growth. Online education offers the flexibility of learning at any time and from anywhere, breaking free from the constraints of traditional learning environments, and granting access to a vast array of educational resources. However, it also presents certain challenges. One notable challenge is the inherent separation between students and teachers in the virtual field of online learning.( 8 ) This physical divide makes it arduous for teachers to gauge the level of student engagement in the learning process, a difficulty that becomes increasingly pronounced as the number of learners rises.( 14 ) Compared to face-to-face instruction, the spatial and temporal detachment in online learning hinders effective communication and interaction between learners and educators, giving rise to a recurring sense of emotional disconnection. This emotional disconnect significantly impacts learners’ online educational experiences and their subsequent outcomes. Therefore, from a pedagogical standpoint, it becomes imperative for educators to automatically discern students’ emotional engagement levels during online learning, furnish timely feedback, and proactively undertake necessary measures to actively involve students in the learning journey. As per the theory of learner engagement, learner engagement stands as the most effective predictor of student development, the level of learner engagement and emotions share a close association with academic performance.( 11 )( 12 ) Several studies have demonstrated that learner engagement correlates with the extent of psychological investment in activities and can serve as a reliable predictor of learning outcomes.( 23 ) Presently, the widely accepted definition of learner engagement, proposed by Fredricks in 2004, encompasses three dimensions: emotional engagement, behavioral engagement, and cognitive engagement. Among these dimensions, emotional engagement pertains to the degree and nature of learners’ positive or negative emotional responses to teachers, peers, school, and academics.( 34 ) Learners who experience a sense of enjoyment tend to be more motivated in tackling challenging problems.( 12 ) Behavioral engagement focuses on learners’ active involvement in social, academic, and extracurricular activities throughout their educational journey, emphasizing quantity over quality in terms of engagement in learning activities.( 34 ) Cognitive engagement relates to the level of knowledge construction during the learning process.( 30 ) Notably, Pekrun et al.’s research suggests that emotional engagement serves as a prerequisite for both cognitive and behavioral engagement. In the context of online learning, the analysis and feedback regarding learners’ emotional engagement assume a critical role. This is because learners’ emotional engagement can serve as an indicator of their willingness to learn, their needs, and their motivation throughout the learning process.( 15 ) Experienced educators can monitor students’ engagement by observing their facial expressions during instruction and adapt their teaching strategies and content accordingly. Facial expressions serve as indicators of a person’s emotional engagement state. Considering the limited sustained attention span of typical students, the level of emotional engagement tends to fluctuate at different stages during a class. Attention span refers to the duration of time an individual can concentrate on a task.( 37 ) Wilson and Korn’s literature review highlighted that students’ attention tends to decline after approximately 10–15 minutes.( 9 ) Several studies have investigated attention span, exploring various aspects such as the relationship between note-taking quantity and attention span( 10 )( 16 )( 23 ), the correlation between the amount of retained information in students’ memory and lecture duration,( 22 ) and the connection between attention span and heart rate per minute.( 13 ) Guo’s research indicated that students’ engagement remains high for the first 6 minutes when watching online learning videos, but subsequently declines rapidly.( 15 ) Therefore, the implementation of a feedback system that automatically analyzes learners’ emotional engagement at different time intervals can assist teachers in summarizing their teaching plans and promptly updating their instructional strategies. From a methodological standpoint, researchers have traditionally relied on manual coding and conventional machine learning methods to identify learners’ emotional engagement in online learning. However, manual coding of datasets is a time-consuming process and is often plagued by issues such as sample imbalance and limited sample size. Furthermore, traditional machine learning methods lack robustness, which has impeded both theoretical and practical advancements in this field. In recent years, the Vision Transformer-based network models have become the state-of-the-art technology in image processing technology, and have made revolutionary achievements in image classification. They address many of the limitations associated with traditional approaches. However, the application of Vision Transformer-based models for detecting learners’ emotional engagement in online learning has not been fully optimized or extensively explored. Moreover, the development of Vision Transformer-based detection and feedback systems specifically tailored to the context of online learning is still needed. This presents challenges in the field of educational research and practice, as researchers and educators strive to leverage the potential of these state-of-the-art technologies. Consequently, this study seeks to accomplish several objectives: ( 1 ) Assess the capability of an optimized Vision Transformer model to infer emotional engagement from facial images captured by a camera. ( 2 ) Investigate the notable variations in emotional engagement among learners at different stages of the online learning process. ( 3 ) Explore the relationship between emotional engagement and learning outcomes. These studies will offer educators and learners valuable methodological and theoretical insights, enhancing their understanding of the significance of emotional engagement in promoting effective learning. Methods 1. Research background and participants This study was recruited at a university in western China, involving 40 junior undergraduate students ( \(\:{M}_{age}=20.9\) ) from various majors, excluding psychology and Marxist philosophy. The participants, consisting of 20 males and 20 females, provided informed consent after a thorough explanation of the study. The recruitment for the experiment began on July 12, 2025 and ended on July 19, 2025. Informed consent forms were distributed to all 40 volunteers who participated in the experiment, and all 40 volunteers agreed and signed the informed consent forms. This is an informed consent form signed by a volunteer in Fig. 1 . The use of data and facial information in this experiment has been agreed and approved by all volunteers. To ensure the legitimacy of this study, the authors submitted a research report and application for ethical review to the local ethics committee of their institution, which subsequently reviewed the application. This review process, including the methods and procedures used in the study and the experiments involving the participants, was ethically sound. The following is the Ethics committee approval of the experimental ethics and the rationality of the research process for this study in Fig. 2 . All experimental procedures were carried out in accordance with relevant guidelines and regulations, and complied with the principles outlined in the Declaration of Helsinki. The following is a statement that the experimental process complies with standards and requirements and has been inspected and verified by the unit where it is located in Fig. 3 . A dedicated laboratory setting was prepared to ensure an uninterrupted environment for the participants during their involvement in the study. The lighting conditions in the laboratory were not manipulated and comprised natural light from both indoor fluorescent lamps and outdoor sunlight. To capture facial video data of the participants during their online learning sessions, a computer equipped with a high-definition camera was set up in the laboratory. The participants’ online learning processes were recorded using the EV screen recording software, combined with the high-definition camera. For the experimental phase, three approximately 10-minute instructional videos were selected from the Chinese University MOOC website. The videos were titled ‘The Psychology of Love’ ‘Innovative Thinking Behind Open Minds’ and ‘Fundamental Principles of Marxism’. All three online courses were classified as national quality courses offered by the Chinese University MOOC. Corresponding test questions were designed for each course to evaluate the participants’ learning outcomes. The test questions we utilized were carefully selected from the supplementary test materials provided after the MOOC courses. These test questions were evaluated by two experts in the respective field of the course, who confirmed that they accurately reflect students' learning outcomes. The test questions are scored out of 10 and consist of four multiple-choice questions, two fill-in-the-blank questions, and one short-answer question. Participants were required to complete the respective test questions after watching each video to obtain their final test scores. The overall data collection process is illustrated in Fig. 4 . In this data collection experiment, a total of 120 segments of online learning videos, each approximately 10 minutes in length, were collected. Building on the research conducted by Whitehill et al., which compared the usefulness of video-based sequences and image-based methods in recognizing engagement levels, this study found that image-based methods had relatively higher accuracy compared to video-based methods. This suggests that engagement is more of a spatial concept rather than a spatiotemporal one.( 21 ) Based on these researches, we obtained a total of 71,185 images for further experimentation. Table 1 presents the number and proportion of images associated with each engagement level. Table 1 Distribution of the number of images for three levels of emotional engagement. Learn emotional engagement Highly engaged Moderately engaged Disengaged Label 3 2 1 Number of pictures 16515 38468 16202 The proportion of the number of pictures 23.20% 54.04% 22.76% 2. Research design The research design consists of seven stages to address the objectives and research questions. Here is a detailed description of each stage. Stage 1: Facial data is obtained from online learning environments using a webcam and stored in a database. Stage 2: The collected data undergoes a cleaning process using Camtasia Studio video editing software to remove any data that does not meet the experimental requirements. This step ensures that only valid and relevant data is retained for further analysis. (Camtasia is a software package produced by TechSmith in the United States that integrates computer screen recording and video editing. It also includes built-in features for Camtasia recorder, Camtasia Studio editor, Camtasia menu maker, Camtasia theater, Camtasia player, and Screencast). Stage 3: Expert coders encode the emotion engagement data based on the theory of emotional engagement. The coders carefully analyze and label the collected data with the appropriate emotion engagement categories, applying their expertise and knowledge in emotional engagement research. Stages 4 and 5: These stages involve the exploration of the first research question. The encoded emotion engagement data is utilized to train and evaluate optimized deep learning models. Through various iterations, the models are refined and adjusted to improve their performance in accurately identifying and classifying emotion engagement in the collected facial data. Stage 6: The trained model with the best parameters is employed to identify and assign emotion engagement labels to unlabeled facial data. This allows for the automatic detection and classification of emotion engagement in previously unlabeled data. Stage 7: Statistical analysis methods, such as Pearson correlation analysis, are utilized to address the second and third research questions. The collected data, including the labeled emotion engagement data and associated learning outcomes, are analyzed to examine the relationships between emotion engagement and learning outcomes. Statistical techniques are employed to determine the strength and significance of these relationships. In the second stage, the cleaning process involves removing video data that does not meet the experimental requirements. Additionally, the videos are segmented into multiple video segments, ensuring that each segment contains only one category of emotion engagement. Camtasia Studio video editing software is employed for this purpose. Invalid video segments, where the learner’s face is obscured or cannot be detected, are excluded during the segmentation process. Figure 5 illustrates the process of video segmentation using Camtasia Studio software, and Fig. 6 displays the results of the video segmentation. In total, 1067 valid video segments were extracted from the initial 120 video segments for analysis in this study. We extracted one frame image every 5 frames from each video segment, excluding images that did not correspond to the engagement level of the video segment. The extracted images were then assigned the engagement level corresponding to their respective video segments. For instance, images extracted from highly engaged video segments were also assigned a highly engaged level. After data cleaning and annotation, a total of 71,185 images were obtained. 3. Coding scheme In the field of online learning, learners’ facial expressions generate a substantial volume of data, which poses challenges in terms of the time required for manual coding. To overcome this methodological challenge, we have developed an optimized deep learning model. To effectively train and evaluate this model, we have devised an encoding scheme for emotional engagement levels in online learning, drawing upon the theory of learners’ emotional engagement.( 21 )( 41 ) The encoding scheme, presented in Table 2 , categorizes learners’ emotional engagement into three distinct levels: highly engaged, moderately engaged, and disengaged. Each emotional engagement category is thoroughly described in Table 2 , providing detailed insights into the characteristics and attributes associated with each level of emotional engagement. Table 2 Coding scheme for learning emotional engagement in online learning. Class Head features Eye features Facial expression features Highly engaged Head upright or inclined forward Staring at the screen, eyes unconsciously widening, increased distance between upper and lower eyelids Surprise, joy, focus, enthusiasm, and other positive expressions. Moderately engaged Head generally upright or slightly tilted to the left or right Line of sight positioned within the screen area, eyes open normally, no change in the distance between upper and lower eyelids Calm, neutral and other neutral expressions Disengaged Head not upright and significant tilt to the left or right The line of sight is positioned at the edge of the screen area or outside the screen area, eyes slightly closed or even completely closed, and the distance between the upper and lower eyelids decreases Bored, tired, indifferent, and other negative expressions In the third phase, to ensure the quality and credibility of the dataset constructed for learners’ engagement, a crowdsourcing approach was employed for data annotation. Three students with academic backgrounds in educational technology were recruited as data annotators, and they underwent training to familiarize themselves with the relevant definitions of learners’ engagement states, the annotation tools, and the specific definitions of the three engagement labels. During the training, a portion of annotated data was provided for practice, and discussions and Q&A sessions were organized to address any issues or questions encountered by the annotators. Guidance and clarification were provided to resolve doubts or disagreements and to ensure a consensus among the annotators. Based on the performance of the annotators during training, they were confirmed as data annotators to participate in the annotation task. To ensure the validity and reliability of the data annotation results, this study adopted a reliability verification method proposed by Kaur et al. A consistency check was conducted using Kendall’s coefficient of agreement.( 24 ) The results of Kendall’s coefficient of agreement for the data annotation by all annotators revealed a high level of consistency, with a Kendall’s coefficient of agreement of 0.889 ( \(\:p<0.01\) ). This high reliability and accuracy of the data annotation confirm the validity and suitability of the annotated data for training and evaluating the online learning emotion engagement recognition model. 4. Automatic engagement detection based on Vision Transformer network and transfer learning In the fourth and fifth stages of the study, an analysis of the encoding scheme for the emotional engagement data was conducted, revealing an issue of class imbalance within the collected dataset. Additionally, due to the smaller number of participants and a larger number of training samples per participant, there was limited diversity in the data, leading to a smaller intra-class distance and a larger inter-class distance. To address these challenges and improve the model’s performance, robustness, and generalization capabilities, the study considered the possibility of pretraining the Vision Transformer network model using the DAiSEE dataset.( 19 ) The pretrained model’s weights would then be utilized as the initial weights for furt19her training using the self-built emotional engagement dataset. By leveraging the pretrained model and incorporating it into the training process, it was anticipated that the model’s performance and generalization abilities could be enhanced, leading to improved results in recognizing and classifying emotional engagement in the online learning context. This approach aimed to address the issue of limited data diversity and enhance the overall effectiveness of the model. The Vision Transformer network model, introduced by Dosovitskiy et al. in 2020, is a notable innovation that adapts the Transformer architecture, originally designed for natural language processing tasks, to the field of computer vision.( 29 ) This model represents a self-attention-based approach to image classification. In contrast to traditional convolutional neural networks, the Vision Transformer does not employ convolutional layers but instead relies exclusively on self-attention mechanisms to extract relevant features from images. The architecture of the Vision Transformer model is visualized in Fig. 7 , showcasing the arrangement of self-attention layers and feed-forward neural networks. Through the use of self-attention mechanisms, the model captures dependencies between different regions of an image, enabling it to effectively process and understand the visual information. This innovative approach has shown promising results in various computer vision tasks and has the potential to significantly impact the field of image classification. To capture more comprehensive and detailed feature information, the Vision Transformer model employs a multi-head self-attention mechanism. This mechanism involves running multiple self-attention mechanisms simultaneously, and then combining their outputs through concatenation and linear transformation to achieve the desired output dimensionality. The calculation formulas for the multi-head self-attention are provided in Equations ( 1 ) and ( 2 ): $$\:MultiHead\left(Q,K,V\right)=Concat\left({head}_{1},\cdots\:,{head}_{h}\right){W}^{O}$$ 1 where \(\:Q\) , \(\:K\) , and \(\:V\) represent the query vector matrix, key vector matrix, and value vector matrix, respectively. The MultiHead function concatenates the outputs of each individual self-attention head, denoted as \(\:{head}_{i}\) , for \(\:i=1\dots\:h\) (the total number of heads). \(\:{W}^{O}\) is the weight matrix used for linear transformation. Each self-attention head \(\:{head}_{i}\) performs the following calculations: $$\:{head}_{i}=Attention\left({Q\cdot\:W}_{i}^{Q},{K\cdot\:W}_{i}^{K},{V\cdot\:W}_{i}^{V}\right)$$ 2 where \(\:{W}_{i}^{Q}\) , \(\:{W}_{i}^{K}\) , and \(\:{W}_{i}^{V}\) are the learnable weight matrices for the query, key, and value projections of the \(\:i\) th self-attention head. The \(\:Attention\) function computes the attention scores and applies them to the values to obtain the attended output. During the training process, we first pre-trained the Vision Transformer network model on the DAiSEE dataset, and then fine-tuned it on our self-built dataset of learners’ learning engagement. 5. Data analysis and automated feedback model To address the first research question, a 10-fold cross-validation method was employed to compare the performance of the Vision Transformer with the baseline model. The parameter settings for the Vision Transformer network model in this study are summarized in Table 3 . These settings were chosen to train the Vision Transformer model effectively and optimize its performance for the task of emotional engagement recognition in online learning. Table 3 Parameter settings for Vision Transformer network model. Parameters Parameter settings Definition Learning rate 0.001 Learning rate Input image size 100×100 Image input size of the model Optimizer Adam Gradient optimization algorithm epochs 500 Number of model iterations Batch size 64 Number of samples per batch in one training iteration Patch size 5×5 Size dimensions of image segmentation blocks To address the second research question, an automatic detection and feedback system was developed. The process flowchart of the system is illustrated in Fig. 8 . This system facilitated the analysis of learner engagement recognition and variations during online learning. The analysis results were communicated to teachers in a timely manner, enabling them to better understand and respond to learners’ engagement levels. In the seventh stage, Pearson correlation analysis was employed to investigate the relationship between learners’ emotional engagement and their learning outcomes, providing insights into the impact of emotional engagement on learning effectiveness. Results 1. Optimized Vision Transformer model identify learners’ emotional engagement in online learning The purpose of this experiment was to evaluate the performance of the optimized Vision Transformer model and transfer learning model in detecting emotional engagement. Firstly, the Vision Transformer network model was pre-trained using the DAiSEE dataset to enhance its feature representation and generalization ability. To assess the impact of transfer learning on model performance, comparative experiments were conducted to evaluate the accuracy of models with and without pre-training. The term ‘without pre-training’ refers to the Vision Transformer network model trained directly on the self-built learning engagement dataset without prior pre-training on the DAiSEE dataset. The results of the comparative experiments are presented in Table 4. Table 4. Comparison of experimental results (%) before and after transfer learning. Accuracy Macro-Recall Macro-Precision Macro-F1 Vision Transformer 91.79 1.54 91.48 1.72 93.04 1.46 91.99 1.55 Vision Transformer +transfer learning 93.82 1.20 93.78 1.04 94.26 0.80 93.95 1.12 From Table 4, it can be observed that the model achieves higher recognition accuracy after transfer learning. This improvement can be attributed to the fact that the pre-training dataset (DAiSEE) for the Vision Transformer model also includes learning engagement. The facial features extracted from the DAiSEE dataset are similar to those extracted from the self-built dataset, thereby significantly increasing the quantity and diversity of training data. As a result, the model’s accuracy in recognizing learning engagement is enhanced. Additionally, we compared the classification performance of our proposed optimized Vision Transformer + transfer learning model with other models in the task of emotional engagement detection. The comparison methods were as follows: Gabor+SVM.(42) Decision tree,(41) ResNet+TCN.(26) LDP-KPCA-DBN.(17) The algorithm is described in Section 2.2. Table 5 presents the comparison of the performance results on the self-built dataset between our algorithm and the baseline models. Table 5. Classification results (%) of emotional engagement. Class Gabor+SVM Decision Tree ResNet+TCN LDP-KPCA-DBN VisionTransformer +transfer learning Highly engaged 77.52 1.61 89.22 1.13 89.93 1.72 95.52 1.68 95.97 1.01 Moderately engaged 73.65 1.82 73.17 1.46 82.24 1.16 85.83 1.41 92.46 0.79 Disengaged 82.19 1.81 83.15 1.72 84.02 1.19 83.39 1.02 93.03 1.29 Mean 77.78 1.52 81.84 0.94 85.40 1.29 88.25 1.13 93.82 1.20 From Table 5, it can be observed that the pre-training results of the Vision Transformer network model on the DAiSEE dataset outperform the accuracy of the other three baseline model based on the same dataset. Taking the LDP-KPCA-DBN baseline algorithm as an example, our algorithm (Vision Transformer and Vision Transformer + transfer learning) outperforms it by 3.54% and 5.57% respectively. This indicates that the Vision Transformer model exhibits excellent recognition performance in the field of computer vision and performs better in recognizing learning engagement compared to conventional neural networks. 2. Students’ engagement vary during online learning Figure 9 illustrates the overall automatic recognition results of learners’ engagement. The results of the self-built dataset of emotional engagement reveal interesting patterns throughout the online course. It can be observed from the figure that the majority of learners’ engagement is categorized as ‘moderate engagement’ throughout the entire duration of the course. However, as the learning session progresses, there is a gradual decline in learners’ engagement. The proportion of ‘high engagement’ and ‘moderate engagement’ decreases, while the proportion of ‘low engagement’ increases. Notably, around the 6-minute mark, there is a more pronounced increase in the proportion of ‘low engagement’, indicating a decline in learners’ overall engagement during that period. Interestingly, towards the end of the course, around the 8-minute mark, there is a slight improvement in learners’ engagement, despite the course nearing its conclusion. This suggests a potential rebound or revitalization of engagement towards the end of the learning session. These insights provide essential information about the fluctuations in learners' engagement levels during online learning and highlight specific time periods when learners may experience changes in their level of engagement. Understanding these patterns can be valuable for educators and instructional designers to optimize learning experiences and implement interventions to sustain and enhance learner engagement throughout the course. Figure 10 presents the overall manual annotation results of learners’ engagement during online learning. The experimental results demonstrate the effectiveness of our proposed online learning sentiment recognition model in accurately identifying and categorizing learners’ engagement levels. The manual annotation results are generally consistent with the automated detection results, indicating that our algorithm is effective in capturing the overall patterns of learners’ changes throughout the online learning process. The alignment between manual annotations and automated detection results allows for a deeper analysis of consistent patterns in learners’ engagement. This highlights the robustness and reliability of our algorithm in providing timely and accurate understanding of learners’ engagement states. Moreover, the effectiveness of our algorithm becomes particularly evident when comparing it to the automated detection and feedback processes. Our algorithm provides a comprehensive and nuanced understanding of learners’ engagement, surpassing the limitations of automated detection alone. It enables us to gain valuable insights into the dynamics of learners’ engagement, facilitating timely and effective interventions to enhance the learning experience. Overall, the experimental results from Figure 10 affirm the effectiveness of our online learning sentiment recognition model in identifying learners’ engagement during online teaching. This supports the notion that our algorithm is a valuable tool for gaining a deeper understanding of student states and optimizing the online learning environment. 3. The relationship between emotional engagement and learning outcomes? In Table 6, the statistical analysis results are presented for the automatic detection results, manual annotations, average of manual annotations, and test results. The metrics included in the table are the mean and standard deviation (SD). For the automatic recognition results, the mean values on three online courses are 2.470,1.931,1.651, respectively, indicating the average emotional engagement level of all learners as determined by the online learning emotion recognition model. The SD represents the variability in the recognition results. The manual annotations by three annotators provide an additional perspective on learners’ emotional engagement. The average of manual annotations combines the individual annotations from the three annotators, providing a more comprehensive assessment of learners’ emotional engagement. Lastly, the test scores represent the learners’ performance on the quiz questions designed to assess their learning effectiveness. The mean and SD values of the test results are provided. These statistical analysis results offer valuable insights into the agreement between the automatic recognition results and manual annotations, as well as the learners’ learning outcomes. They provide a quantitative assessment of the emotional engagement levels, helping to validate the effectiveness of the online learning emotion recognition model and evaluate the learners’ understanding and retention of the video content. Table 6. Statistical analysis of emotional engagement and learning outcomes. Course ID Annotator1 Annotator2 Annotator3 Average of manually annotated results Automatic detection Test score 1 Mean 2.782 2.224 2.565 2.523 2.470 8.100 SD 0.323 0.329 0.410 0.377 0.323 1.370 N 40 40 40 120 40 40 2 Mean 2.10 1.98 1.89 1.986 1.931 6.300 SD 0.201 0.216 0.260 0.226 0.245 0.823 N 40 40 40 120 40 40 3 Mean 1.73 1.80 1.63 1.716 1.651 4.400 SD 0.122 0.174 0.158 0.175 0.121 0.516 N 40 40 40 120 40 40 Table 7 presents the results of the Pearson correlation analysis conducted to examine the relationship between learners’ emotional engagement and their learning outcomes. The variables analyzed include emotional engagement through manual annotation and automatic detection, as well as learning outcomes. Whether it is automatic emotion recognition or manual annotation results, there is a significant relationship with the learning outcomes. Specifically, the coefficient correlations between automatic detection and test score were r =0.860**,0.664*, and 0.707*. The coefficient correlations between manual annotation and test score were r =0.799**,0.657*, and 0.636*. This suggests that the correlation between emotional engagement and learning outcomes is statistically significant. The results of the Pearson correlation analysis support the hypothesis that learners’ emotional engagement has a positive influence on their learning outcomes. This experimental result emphasizes the importance of emotional engagement in online learning and suggests that learners who are more emotionally engaged tend to achieve better learning outcomes. Overall, the results of the Pearson correlation analysis provide evidence of a significant and positive correlation between learners’ emotional engagement and their learning outcomes, highlighting the relevance of emotional engagement in the context of online learning. Table 7. The Pearson correlation analysis between test score, automatic detection and manual annotation. Course ID Manual annotation Automatic detection Test score 1 Manual annotation 1 Automatic detection 0.869** 1 Test score 0.799** 0.860** 1 2 Manual annotation 1 Automatic detection 0.876** 1 Test score 0.657* 0.664* 1 3 Manual annotation 1 Automatic detection 0.887** 1 Test score 0.636* 0.707* 1 Note. N=40, * p <0.05, ** p <0.01, ** p <0.001 Discussion In this study, we conducted an analysis of the emotional engagement of 40 college students in e-learning courses. By developing emotional engagement detection and feedback models, we examined the dynamics of students’ emotional engagement throughout the online learning process and investigated whether there were significant variations in emotional engagement across the course. Additionally, we explored the relationship between changes in emotional engagement and academic achievement. To address the challenges of limited sample diversity and sample imbalance in facial analysis during online learning, we developed an optimized emotional engagement detection model using the Vision Transformer architecture and transfer learning techniques. From the results presented in Table 4 and Table 5 , it is evident that our proposed emotional engagement detection model outperformed existing models in terms of classification accuracy. Compared to traditional machine methods and shallow neural networks that require manual feature design, our model achieved significant improvements in performance. Specifically, our model demonstrated an improvement of 3.54% and 5.57% in classification accuracy compared to existing emotional engagement detection models. This improvement highlights the effectiveness of our approach, which utilizes Vision Transformer and transfer learning techniques to capture richer contextual information and enhance the model’s generalization capabilities. The analysis of learners’ emotional engagement revealed interesting patterns. Figure 9 highlighted that learners generally exhibited a state of moderate engagement in the initial minutes of the course. However, as time progressed, their engagement gradually declined, with a sharp drop around the six-minute mark, leading to disengagement in some learners. This research result emphasizes the importance of capturing learners’ attention and motivation early in the learning process. Educators should incorporate engaging activities and content at the beginning to foster enthusiasm and enhance learners’ motivation.( 40 ) Notably, towards the end of the course, there was a slight recovery in learners’ emotional engagement, potentially due to the awareness that the course was nearing its conclusion. This observation suggests that learners may become more engaged when they realize the final stage of the course is approaching. The statistical analysis results presented in Table 6 and Table 7 demonstrated a positive correlation between learners’ emotional engagement and their learning effectiveness. Both the automated recognition results and manual annotations indicated that learners’ engagement positively influenced their learning outcomes. When learners were interested and fully engaged, their learning effectiveness tended to be higher. On the other hand, if learners experienced boredom or lacked full engagement, their learning effectiveness was negatively impacted. These research results highlight the importance of monitoring and addressing learners’ emotional engagement during teaching. Teachers should strive to introduce stimulating content and activities to enhance learners’ engagement and improve learning effectiveness. The positive correlation between emotional engagement and learning effectiveness aligns with the understanding that emotions play a crucial role in driving learning outcomes. Emotions play a significant role in learning achievement. This research result is consistent with previous research emphasizing the impact of emotions on education.( 17 ) Conclusions Given the paramount importance of the interaction between emotional engagement and learning outcomes, this study employs an optimized deep learning model and employed statistical analysis methods to explore the complex relationships between them. We collected facial data from 40 college students during their online learning to come up with a dataset containing 71,185 instances of emotional engagement. On this basis, an automatic detection and feedback model for emotional engagement is constructed. The results of this study are twofold. First, compared to existing research in the field, the developed model shows a significant improvement in recognition performance. This marks a significant breakthrough in accurately identifying and interpreting emotional engagement in online learning environments. Second, Pearson correlation analysis revealed a significant relationship between emotional engagement and learning outcomes. Specifically, those learners who demonstrate genuine interest and total commitment to the learning material generally achieve higher academic achievement. Conversely, individuals who experience apathy or struggle to fully engage in the learning process often show impaired learning outcomes. Based on these studies, several suggestions to foster student motivation and optimize learning outcomes can be offered. Firstly, educators can curate captivating and intellectually stimulating discussion tasks to engender a comprehensive emotional engagement among learners. This approach fosters deep cognitive involvement and enhances the overall learning experience.( 12 )( 35 ) Secondly, the machine learning-based automated learning intervention techniques has shown great promise in bolstering online learning outcomes.( 20 ) By incorporating automated components within online learning platforms, educators can dynamically adjust and optimize the level of emotional engagement among learners. Finally, it is recommended that the broad potential of mood monitoring devices be fully utilized in educational institutions. These devices can provide valuable insights into students’ emotions, helping teachers gather feedback and adjust teaching methods accordingly. Declarations Funding Funding: National Natural Science Foundation of China (No. 62177032), “Research on the Autonomous Training and Evaluation Model for Pre-service Teachers’ Classroom Teaching Expression Competence.” Author contributions statement Guanyu Chen is responsible for article writing and revision, as well as communication work. Guangxin Han is responsible for data calculation and processing, as well as icon creation and modification. Juan Niu has provided academic research and theoretical support for this study. Juhou He is the administrator of this project and provides guidance throughout the research process. Data availability statement The data involved in this study are public. If you need them, please contact the author by email. The author will provide the complete data and related information. Additional information Competing interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. References Ludjon Roshi, E-learning Statistics 2023:Online Learning Facts[EB/OL]. https://codeless.co/elearning-statistics/, 2023. Gemma Josep. 5 reasons why Online learning is the future of Education in 2023[EB/OL].https://www.educations.com/articles-and-advice/5-reasons-online-learning-is-future-of-education-17146, 2023. Means, Barbara and Toyama, Yuki and Murphy, Robert and Bakia, Marianne and Jones, Karla. Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies. Project Report. Centre for Learning Technology[M]. Association for learning technology, 2009(8). Joshua Gans. What is the future of online education?[EB/OL]. https://www.weforum.org/agenda/2014/12/what-is-the-future-of-online-education/, 2014. 中国互联网络信息中心. 第50次中国互联网网络发展状况统计报告[EB/OL]. https://www.cnnic.net.cn/n4/2022/0914/c88-10226.html, 2022-08-31. 祝智庭, 胡姣. 技术赋能后疫情教育创变: 线上线下融合教学新样态[J]. 开放教育研究, 2021, 27(1): 13-23. 刘司卓,李爽,黄嘉靖.直播课学习行为投入评价的实证研究[J].中国远程教育,2021(2):36-45,58. Venton B J, Pompano R R. Strategies for enhancing remote student engagement through active learning[J]. Analytical and Bioanalytical Chemistry, 2021, 413(6): 1507-1512. Sümer Ö, Goldberg P, D'Mello S, et al. Multimodal engagement analysis from facial videos in the classroom[J]. IEEE Transactions on Affective Computing, 2021. 魏艳涛, 雷芬, 胡美佳, 等. 学生表情识别研究综述[J]. 中国教育信息化, 2020(21):48-55. Baker R S, D'Mello S K, Rodrigo M M T, et al. Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments[J]. International Journal of Human-Computer Studies, 2010, 68(4): 223-241. D’Mello S, Lehman B, Pekrun R, et al. Confusion can be beneficial for learning[J]. Learning and Instruction, 2014, 29: 153-170. Jagers R J, Rivas-Drake D, Williams B. Transformative social and emotional learning (SEL): Toward SEL in service of educational equity and excellence[J]. Educational Psychologist, 2019, 54(3): 162-184. Pabba C, Kumar P. An intelligent system for monitoring students' engagement in large classroom teaching through facial expression recognition[J]. Expert Systems, 2022, 39(1):e12839.1-e12839.28. 曹晓明, 张永和, 潘萌, 等. 人工智能视域下的学习参与度识别方法研究——基于一项多模态数据融合的深度学习实验分析[J]. 远程教育杂志, 2019, 37(1): 32-44. Bosch N, Chen Y, D’Mello S. It’s written on your face: detecting affective states from facial expressions while learning computer programming[C]. Intelligent Tutoring Systems: 12th International Conference, ITS 2014, Honolulu, HI, USA, June 5-9, 2014 Proceedings 12. Springer, 2014:39-44. Dewan M, Murshed M, Lin F. Engagement detection in online learning: a review[J]. Smart Learning Environments, 2019, 6(1): 1-20. 张琪, 武法提. 学习分析中的生物数据表征——眼动与多模态技术应用前瞻[J]. 电化教育研究, 2016, 37(9): 76-81. D'Mello S K, Craig S D, Graesser A C. Multimethod assessment of affective experience and expression during deep learning[J]. International Journal of Learning Technology, 2009, 4(3-4): 165-187. D’Mello S K, Graesser A. Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features[J]. User Modeling and User-Adapted Interaction, 2010, 20: 147-187. Whitehill J, Serpell Z, Lin Y-C, et al. The faces of engagement: Automatic recognition of student engagementfrom facial expressions[J]. IEEE Transactions on Affective Computing, 2014, 5(1): 86-98. Kamath A, Biswas A, Balasubramanian V. A crowdsourced approach to student engagement recognition in e-learning environments[C]. 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2016:1-9. Gupta A, D'Cunha A, Awasthi K, et al. Daisee: Towards user engagement recognition in the wild[J]. arXiv preprint arXiv:160901885, 2016. Kaur A, Mustafa A, Mehta L, et al. Prediction and localization of student engagement in the wild[C]. 2018 Digital Image Computing: Techniques and Applications (DICTA). IEEE, 2018:1-8. Mukhopadhyay M, Pal S, Nayyar A, et al. Facial emotion detection to assess Learner's State of mind in an online learning system[C]. Proceedings of the 2020 5th international conference on intelligent information technology. 2020:107-115. Abedi A, Khan S S. Improving state-of-the-art in detecting student engagement with resnet and tcn hybrid network[C]. 2021 18th Conference on Robots and Vision (CRV). IEEE, 2021:151-157. 王嘉豪, 徐敏, 孙众, 等. 数据分布不平衡的课堂参与度自动识别研究[J]. 小型微型计算机系统: 1-8. Wei Q, Sun B, He J, et al. BNU-LSVED 2.0: Spontaneous multimodal student affect database with multi-dimensional labels[J]. Signal Processing: Image Communication, 2017, 59: 168-181. Bian C, Zhang Y, Yang F, et al. Spontaneous facial expression database for academic emotion inference in online learning[J]. IET Computer Vision, 2019, 13(3): 329-337. Schmieder A. A glossary of educational reform[J]. Journal of Teacher Education, 1973, 24(1): 55-62. Fisher C W, Berliner D C, Filby N N, et al. Teaching behaviors, academic learning time, and student achievement: An overview[J]. The Journal of classroom interaction, 1981, 17(1): 2-15. Skinner E A, Belmont M J. Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year[J]. Journal of educational psychology, 1993, 85(4): 571-581. Connell J P, Wellborn J G. Competence, Autonomy, and Relatedness: A motivational analysis of self-system processes[J]. Journal of Personality and Social Psychology, 1991(65):43-77. Fredricks J A, Blumenfeld P C, Paris A H. School engagement: Potential of the concept, state of the evidence[J]. Review of educational research, 2004, 74(1): 59-109. Pekrun R, Lichtenfeld S, Marsh H W, et al. Achievement emotions and academic performance: Longitudinal models of reciprocal effects[J]. Child development, 2017, 88(5): 1653-1670. Kahu E R, Nelson K. Student engagement in the educational interface: Understanding the mechanisms of student success[J]. Higher education research & development, 2018, 37(1): 58-71. Alyuz N, Okur E, Oktay E, et al. Towards an emotional engagement model: Can affective states of a learner be automatically detected in a 1:1 learning scenario?[C]. 24th ACM Conference on User Modeling, Adaptation and Personalization (UMAP). 2016. Aslan S, Mete S E, Okur E, et al. Human expert labeling process (HELP): towards a reliable higher-order user state labeling process and tool to assess student engagement[J]. Educational Technology, 2017: 53-59. Sidney K D, Craig S D, Gholson B, et al. Integrating affect sensors in an intelligent tutoring system[C]. Affective Interactions: The Computer in the Affective Loop Workshop at. 2005:7-13. 李振华. 融入多源数据的在线学习投入测评方法研究[D]. 华中师范大学, 2020. Kuh G D . The national survey of student engagement: Conceptual and empirical foundations[J]. New Directions for Institutional Research, 2010, 2009(141):5-20. Whitehill J, Serpell Z, Lin Y C, et al. The faces of engagement: Automatic recognition of student engagement from facial expressions[J]. IEEE Transactions on Affective Computing, 2014,(1):86-98. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 07 Jan, 2026 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 07 Nov, 2025 Reviews received at journal 05 Nov, 2025 Reviews received at journal 30 Oct, 2025 Reviewers agreed at journal 30 Oct, 2025 Reviewers agreed at journal 29 Oct, 2025 Reviewers invited by journal 29 Oct, 2025 Editor assigned by journal 22 Oct, 2025 Editor invited by journal 19 Sep, 2025 Submission checks completed at journal 19 Sep, 2025 First submitted to journal 19 Sep, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7581467","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":532435470,"identity":"f4a08a17-e34a-4543-af20-27fe408d061e","order_by":0,"name":"Guanyu Chen","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAApUlEQVRIiWNgGAWjYHACNoYPUJYEkRqY2RhnkKyFmYckLbrz+489tvlzJ8/gAPPB2zwMdnkEtZgdY2Y3zm17VmxwgC3ZmochuZgYLWzSuQ2HEzcc4DGT5mE4kNhAlBaLPyAt/N9I0MLABraFjVgtyWaSvW2HE2ceZjO2nGOQTISWwwefSfwAOqzvePPDG28q7AhrQQBmEGFAvPpRMApGwSgYBXgAAAJSOJvoKO8rAAAAAElFTkSuQmCC","orcid":"","institution":"Shaanxi Normal University","correspondingAuthor":true,"prefix":"","firstName":"Guanyu","middleName":"","lastName":"Chen","suffix":""},{"id":532435471,"identity":"adff586c-c5a8-4c79-825e-c91d1126784c","order_by":1,"name":"Guangxin Han","email":"","orcid":"","institution":"Shaanxi Normal University","correspondingAuthor":false,"prefix":"","firstName":"Guangxin","middleName":"","lastName":"Han","suffix":""},{"id":532435472,"identity":"16c45f1a-bcbb-4e03-bca4-7a71c6c52cb9","order_by":2,"name":"Juan Niu","email":"","orcid":"","institution":"Shaanxi Normal University","correspondingAuthor":false,"prefix":"","firstName":"Juan","middleName":"","lastName":"Niu","suffix":""},{"id":532435473,"identity":"3e4fad98-8f11-4038-a5a5-7c3ceae65d71","order_by":3,"name":"Juhou He","email":"","orcid":"","institution":"Shaanxi Normal University","correspondingAuthor":false,"prefix":"","firstName":"Juhou","middleName":"","lastName":"He","suffix":""}],"badges":[],"createdAt":"2025-09-10 09:53:23","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7581467/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7581467/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-025-34871-x","type":"published","date":"2026-01-07T15:57:59+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":94050490,"identity":"0583c0df-1826-4e49-9786-7a334db1aef8","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":2760221,"visible":true,"origin":"","legend":"","description":"","filename":"ManuscriptScientficreports.docx","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/bc096b7eb9846b5a34420109.docx"},{"id":94051020,"identity":"17ef8755-b694-4e9d-8e71-b967f9c81028","added_by":"auto","created_at":"2025-10-21 23:55:28","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5602,"visible":true,"origin":"","legend":"","description":"","filename":"c79e15c7f07b4978abab6190deb53625.json","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/4d0a0a44aa7d52b8db479f68.json"},{"id":94051381,"identity":"3174e2ba-0b79-4f66-850a-379703d365f7","added_by":"auto","created_at":"2025-10-22 00:03:28","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":159402,"visible":true,"origin":"","legend":"","description":"","filename":"c79e15c7f07b4978abab6190deb536251enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/68fa1832df0e346103327fac.xml"},{"id":94050487,"identity":"4e7aac6a-6dfd-419b-aca6-d037f16faf0c","added_by":"auto","created_at":"2025-10-21 23:47:27","extension":"jpeg","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":375910,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/9d49d982d36853b8891e5b88.jpeg"},{"id":94051382,"identity":"1f434bf7-eb96-4aca-9543-f482e7088702","added_by":"auto","created_at":"2025-10-22 00:03:28","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":40229,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/1bd7833bf931017faf0e3b4d.png"},{"id":94050504,"identity":"5011d54f-bbf8-44f0-bb23-bbd1cd503d35","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"jpeg","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":375910,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/8744a1e846062d50924fd9c9.jpeg"},{"id":94051032,"identity":"53add653-ce15-45db-8734-7438186aee17","added_by":"auto","created_at":"2025-10-21 23:55:28","extension":"png","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1011576,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage12.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/0d662308523a8d7935461801.png"},{"id":94050495,"identity":"aa1c68ae-0e0f-41e1-b4c3-03d25bb9b59f","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"jpeg","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":418797,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage13.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/06b822d826defca9b38fae72.jpeg"},{"id":94051386,"identity":"2f187796-1239-4f69-9c4e-db897f1a9e9d","added_by":"auto","created_at":"2025-10-22 00:03:29","extension":"jpeg","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":72552,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage14.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/76563cfde3990210e5d88b75.jpeg"},{"id":94050514,"identity":"41f54492-d840-4d64-bbf8-d4602ab90498","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":288839,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage15.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/307fa7449c6f62c496568dfd.png"},{"id":94050498,"identity":"43eb1f2c-5a69-4006-be5a-26d01958907d","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"jpeg","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":241372,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage16.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/b000508c6d809b4aa5e4733b.jpeg"},{"id":94051029,"identity":"86237b5e-13f3-43a6-921f-d7605860c883","added_by":"auto","created_at":"2025-10-21 23:55:28","extension":"jpeg","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":92661,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage17.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/a0c3b09ea9ae7287f4d87c24.jpeg"},{"id":94050497,"identity":"9fbcc80e-dd02-47f1-a30e-26942b408c0f","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"jpeg","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":113413,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage18.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/07a2aec5eec9689bd4206120.jpeg"},{"id":94050506,"identity":"35dbf58d-b6c3-4ca5-baf1-857d490deba5","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":38678,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage19.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/630acffdcb414665ad0df68f.png"},{"id":94050499,"identity":"b9796ddf-69ba-41ed-a0b9-282c8ca4b645","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1011576,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage12.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/7fd5d7bdf45f16361a945ffe.png"},{"id":94050501,"identity":"1e436069-ef78-4a88-8f99-c7fa8e5f1f62","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":15,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":40229,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/1bef22a662ceb50bcef49a3a.png"},{"id":94050505,"identity":"6b998d92-f511-49ea-a904-1e6f9153a2a7","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"jpeg","order_by":16,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":418797,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage13.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/077ceb13293a536cac8ae599.jpeg"},{"id":94050526,"identity":"f450991d-8e97-42f1-a662-07dd5893b57b","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"jpeg","order_by":17,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":72552,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage14.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/58bad6e205d752d53e487d9b.jpeg"},{"id":94050523,"identity":"563a93a2-7ae6-4ab6-b30a-a71b9ead87f9","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":18,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":288839,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage15.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/23c3b23813ee2c12353bd55d.png"},{"id":94050507,"identity":"0628df10-40bf-47bb-9e87-5e81756be7ae","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"jpeg","order_by":19,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":241372,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage16.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/948d49e1fed62dba8b13d81e.jpeg"},{"id":94051036,"identity":"1c5a2285-dd6a-45a1-8612-d3b87164d8dc","added_by":"auto","created_at":"2025-10-21 23:55:29","extension":"jpeg","order_by":20,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":92661,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage17.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/d41a184420397a7fcff9e63c.jpeg"},{"id":94050542,"identity":"e84db436-98c9-4405-b281-b76c42cc8bd2","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"jpeg","order_by":21,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":113413,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage18.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/e4f2466c041722a73960ca0b.jpeg"},{"id":94051030,"identity":"3048501a-2c24-4d86-a7ae-3018032acc6b","added_by":"auto","created_at":"2025-10-21 23:55:28","extension":"png","order_by":22,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":38678,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage19.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/889f6d5ba04ced0ba43978cf.png"},{"id":94051531,"identity":"64cbe93a-91ad-496c-adad-e4a774c2c20f","added_by":"auto","created_at":"2025-10-22 00:11:29","extension":"png","order_by":23,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":362687,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/adcfad79698af22aa9aaafca.png"},{"id":94050517,"identity":"90208775-5ef7-4760-8c96-156fde536faf","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":24,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":10266,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/a84762bcc4782d14078041af.png"},{"id":94050529,"identity":"7b3bee06-6374-400c-b0d4-6bea24a8918e","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":25,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":362687,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/90c78e5f113a045b8e670a1f.png"},{"id":94050540,"identity":"980b33b4-4afe-4ef6-8b33-95e4f0fd56d7","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":26,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":55092,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage12.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/ce2cac470272b55a55460786.png"},{"id":94050503,"identity":"771ffba0-d60b-4b5a-bfc6-59b76cc248cf","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":27,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":384694,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage13.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/4f59a4ed3cb05a69ff5e1ded.png"},{"id":94050532,"identity":"f8fbe9fc-10d5-45b7-ae0d-f969ed17c74c","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":28,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":23813,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage14.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/c4f638bfe52d79f9ce948540.png"},{"id":94050524,"identity":"c0bf9811-48a0-48bf-8d23-f1806b69ef13","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":29,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":36963,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage15.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/435300d8568ca6dd60b0e823.png"},{"id":94050521,"identity":"4acd92a4-44bd-4fdf-9209-0a39a3d3e8a9","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":30,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":145521,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage16.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/9d59adcc6c7d148e8ded50d0.png"},{"id":94051033,"identity":"6e302036-27f4-4351-aaaf-bd68226baee3","added_by":"auto","created_at":"2025-10-21 23:55:28","extension":"png","order_by":31,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":37063,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage17.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/1f54b08d87c93cbf77266f1c.png"},{"id":94050508,"identity":"539aad6a-8d9e-4635-886e-49fbd8d8029a","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":32,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":39582,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage18.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/420514c8dc95d010e67bd09d.png"},{"id":94050509,"identity":"aa49ca1a-3e24-457b-a9e6-838f03eddb85","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":33,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":8556,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage19.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/535d7082db4ad722ffa27382.png"},{"id":94051034,"identity":"94287b1d-2a2b-446b-ac57-bf98ae3cedaf","added_by":"auto","created_at":"2025-10-21 23:55:28","extension":"png","order_by":34,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":55092,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/f80d1c4eb769cda1c4e6fb5d.png"},{"id":94050536,"identity":"0f02e0de-2a5c-4829-b85a-79da6d2d9f64","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":35,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":10266,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage20.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/d4dc441a044fea1b4281651f.png"},{"id":94050513,"identity":"2e8f87a7-e12b-4b25-aa9e-7caf96db8228","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":36,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":384694,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/c0394b30c37a74e8fb4f31de.png"},{"id":94050528,"identity":"4555ba5d-f2b9-4557-993f-8af8fda6befd","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":37,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":23813,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/b9d830048ed2d78e9e20d555.png"},{"id":94050522,"identity":"4e416fd4-4637-4767-8882-84fd4f688854","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":38,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":36963,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/7ed53f67436ac4e4ffd7ac9c.png"},{"id":94050531,"identity":"c274380b-842e-4e9e-b19b-540a9339f50c","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":39,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":145521,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/bfe30daf52904e76896eb42e.png"},{"id":94050534,"identity":"e86024d6-e6a9-4985-81ab-99b49b36ffa1","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":40,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":37063,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage17.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/6fbee549b60868e9ff9511ba.png"},{"id":94050519,"identity":"8748c03c-1645-4ed0-8cb4-001c8cc0d3e8","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":41,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":39582,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage18.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/0a2555889a0b4f44b0eea158.png"},{"id":94050530,"identity":"1347985c-0488-40fd-9292-bea9789282ca","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":42,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":8556,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage19.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/e0c8b7f4664a79bc40db17d7.png"},{"id":94051035,"identity":"998ea8d0-4a10-402a-9fb7-cc365d52be06","added_by":"auto","created_at":"2025-10-21 23:55:29","extension":"xml","order_by":43,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":155478,"visible":true,"origin":"","legend":"","description":"","filename":"c79e15c7f07b4978abab6190deb536251structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/e1048bf0ed5318223abf3baf.xml"},{"id":94051040,"identity":"87fa782a-ed6a-4f12-85e8-98b85df083c0","added_by":"auto","created_at":"2025-10-21 23:55:29","extension":"html","order_by":44,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":174034,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/6951a5ff55c9b369849a2631.html"},{"id":94050488,"identity":"c04315d1-2098-46f5-8050-2e50363a1fb7","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":635853,"visible":true,"origin":"","legend":"\u003cp\u003eInformed consent signed by experimental volunteers\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/fbec1b8b84408e7b5257097d.png"},{"id":94051681,"identity":"7366df0d-f6c0-4ec2-9507-90d52c4dfc29","added_by":"auto","created_at":"2025-10-22 00:19:28","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":1042306,"visible":true,"origin":"","legend":"\u003cp\u003eEthics committee approval of the experimental ethics and the rationality of the research process\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/3f99366f91f854a2434c8bf8.png"},{"id":94051529,"identity":"5e9904f1-dc70-44f0-b281-ef72bfd21e85","added_by":"auto","created_at":"2025-10-22 00:11:28","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":669433,"visible":true,"origin":"","legend":"\u003cp\u003eThe research method was approved\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/5d3b2039e0dc01d863fcc53c.png"},{"id":94050502,"identity":"b5550a58-93ff-44c4-a860-a569d08f2225","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":220109,"visible":true,"origin":"","legend":"\u003cp\u003eFlowchart for constructing a dataset of emotional engagement in online learning\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/43be1e326826a8bbfab2890c.png"},{"id":94050511,"identity":"daf02392-bcc7-4c42-8ee8-7fb21a638183","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":246947,"visible":true,"origin":"","legend":"\u003cp\u003eVideo trimming using Camtasia Studio software.\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/195536d29f7348b02bbd778b.png"},{"id":94051023,"identity":"31f7c20d-2c0d-46e9-8dc0-dc35ab3106ff","added_by":"auto","created_at":"2025-10-21 23:55:28","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":435041,"visible":true,"origin":"","legend":"\u003cp\u003eVideo segmentation results.\u003c/p\u003e","description":"","filename":"6.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/c6af1740c03664139a6007c1.png"},{"id":94050493,"identity":"da9f82b6-a3cf-4d96-b871-54d62efef696","added_by":"auto","created_at":"2025-10-21 23:47:28","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":245217,"visible":true,"origin":"","legend":"\u003cp\u003eVision Transformer model architecture diagram.\u003c/p\u003e","description":"","filename":"7.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/5b767ab51587cc1fe7bcdc48.png"},{"id":94051039,"identity":"39b71790-49a0-4574-a57e-46367a64d892","added_by":"auto","created_at":"2025-10-21 23:55:29","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":298319,"visible":true,"origin":"","legend":"\u003cp\u003eThe automatic detection and feedback system.\u003c/p\u003e","description":"","filename":"8.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/0d49bf57c9b6cfef5e884337.png"},{"id":94050539,"identity":"e05b7582-9fa9-4378-9997-05db80f3aaf8","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":109148,"visible":true,"origin":"","legend":"\u003cp\u003eThe automatic detection results of all learners.\u003c/p\u003e","description":"","filename":"9.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/65e4b7b8e2514f6de094e3ad.png"},{"id":94050527,"identity":"157e4093-1b3e-40bd-8721-830960fad474","added_by":"auto","created_at":"2025-10-21 23:47:29","extension":"png","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":116228,"visible":true,"origin":"","legend":"\u003cp\u003eThe manual annotation results of all learners.\u003c/p\u003e","description":"","filename":"10.png","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/72bd6b04969bd9da93bda174.png"},{"id":100069556,"identity":"510d491a-226d-4bdb-b328-2f061e95f0a4","added_by":"auto","created_at":"2026-01-12 16:14:55","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4578998,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7581467/v1/d0d6341c-940f-46c6-9e20-782d44c25518.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Understanding the Impact of Emotional Engagement on Learning Outcomes in Online Education: An Automated Analysis Approach","fulltext":[{"header":"Introduction","content":"\u003cp\u003eWith the rapid progress and technological advancements in education, internet services have gained widespread adoption and implementation across major universities, as well as primary and secondary schools. Consequently, online education has undergone substantial growth. Online education offers the flexibility of learning at any time and from anywhere, breaking free from the constraints of traditional learning environments, and granting access to a vast array of educational resources. However, it also presents certain challenges. One notable challenge is the inherent separation between students and teachers in the virtual field of online learning.(\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e) This physical divide makes it arduous for teachers to gauge the level of student engagement in the learning process, a difficulty that becomes increasingly pronounced as the number of learners rises.(\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e) Compared to face-to-face instruction, the spatial and temporal detachment in online learning hinders effective communication and interaction between learners and educators, giving rise to a recurring sense of emotional disconnection. This emotional disconnect significantly impacts learners’ online educational experiences and their subsequent outcomes. Therefore, from a pedagogical standpoint, it becomes imperative for educators to automatically discern students’ emotional engagement levels during online learning, furnish timely feedback, and proactively undertake necessary measures to actively involve students in the learning journey.\u003c/p\u003e\u003cp\u003eAs per the theory of learner engagement, learner engagement stands as the most effective predictor of student development, the level of learner engagement and emotions share a close association with academic performance.(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e)(\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e) Several studies have demonstrated that learner engagement correlates with the extent of psychological investment in activities and can serve as a reliable predictor of learning outcomes.(\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e) Presently, the widely accepted definition of learner engagement, proposed by Fredricks in 2004, encompasses three dimensions: emotional engagement, behavioral engagement, and cognitive engagement. Among these dimensions, emotional engagement pertains to the degree and nature of learners’ positive or negative emotional responses to teachers, peers, school, and academics.(\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e) Learners who experience a sense of enjoyment tend to be more motivated in tackling challenging problems.(\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e) Behavioral engagement focuses on learners’ active involvement in social, academic, and extracurricular activities throughout their educational journey, emphasizing quantity over quality in terms of engagement in learning activities.(\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e) Cognitive engagement relates to the level of knowledge construction during the learning process.(\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e) Notably, Pekrun et al.’s research suggests that emotional engagement serves as a prerequisite for both cognitive and behavioral engagement. In the context of online learning, the analysis and feedback regarding learners’ emotional engagement assume a critical role. This is because learners’ emotional engagement can serve as an indicator of their willingness to learn, their needs, and their motivation throughout the learning process.(\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e) Experienced educators can monitor students’ engagement by observing their facial expressions during instruction and adapt their teaching strategies and content accordingly. Facial expressions serve as indicators of a person’s emotional engagement state.\u003c/p\u003e\u003cp\u003eConsidering the limited sustained attention span of typical students, the level of emotional engagement tends to fluctuate at different stages during a class. Attention span refers to the duration of time an individual can concentrate on a task.(\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e) Wilson and Korn’s literature review highlighted that students’ attention tends to decline after approximately 10–15 minutes.(\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e) Several studies have investigated attention span, exploring various aspects such as the relationship between note-taking quantity and attention span(\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e)(\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e)(\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e), the correlation between the amount of retained information in students’ memory and lecture duration,(\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e) and the connection between attention span and heart rate per minute.(\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) Guo’s research indicated that students’ engagement remains high for the first 6 minutes when watching online learning videos, but subsequently declines rapidly.(\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e) Therefore, the implementation of a feedback system that automatically analyzes learners’ emotional engagement at different time intervals can assist teachers in summarizing their teaching plans and promptly updating their instructional strategies.\u003c/p\u003e\u003cp\u003eFrom a methodological standpoint, researchers have traditionally relied on manual coding and conventional machine learning methods to identify learners’ emotional engagement in online learning. However, manual coding of datasets is a time-consuming process and is often plagued by issues such as sample imbalance and limited sample size. Furthermore, traditional machine learning methods lack robustness, which has impeded both theoretical and practical advancements in this field. In recent years, the Vision Transformer-based network models have become the state-of-the-art technology in image processing technology, and have made revolutionary achievements in image classification. They address many of the limitations associated with traditional approaches. However, the application of Vision Transformer-based models for detecting learners’ emotional engagement in online learning has not been fully optimized or extensively explored. Moreover, the development of Vision Transformer-based detection and feedback systems specifically tailored to the context of online learning is still needed. This presents challenges in the field of educational research and practice, as researchers and educators strive to leverage the potential of these state-of-the-art technologies.\u003c/p\u003e\u003cp\u003eConsequently, this study seeks to accomplish several objectives: (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) Assess the capability of an optimized Vision Transformer model to infer emotional engagement from facial images captured by a camera. (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) Investigate the notable variations in emotional engagement among learners at different stages of the online learning process. (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e) Explore the relationship between emotional engagement and learning outcomes. These studies will offer educators and learners valuable methodological and theoretical insights, enhancing their understanding of the significance of emotional engagement in promoting effective learning.\u003c/p\u003e"},{"header":"Methods","content":"\u003ch3\u003e1. Research background and participants\u003c/h3\u003e\u003cp\u003eThis study was recruited at a university in western China, involving 40 junior undergraduate students (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{M}_{age}=20.9\\)\u003c/span\u003e\u003c/span\u003e) from various majors, excluding psychology and Marxist philosophy. The participants, consisting of 20 males and 20 females, provided informed consent after a thorough explanation of the study. The recruitment for the experiment began on July 12, 2025 and ended on July 19, 2025. Informed consent forms were distributed to all 40 volunteers who participated in the experiment, and all 40 volunteers agreed and signed the informed consent forms. This is an informed consent form signed by a volunteer in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. The use of data and facial information in this experiment has been agreed and approved by all volunteers.\u003c/p\u003e\u003cp\u003e To ensure the legitimacy of this study, the authors submitted a research report and application for ethical review to the local ethics committee of their institution, which subsequently reviewed the application. This review process, including the methods and procedures used in the study and the experiments involving the participants, was ethically sound. The following is the Ethics committee approval of the experimental ethics and the rationality of the research process for this study in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e\u003cp\u003e All experimental procedures were carried out in accordance with relevant guidelines and regulations, and complied with the principles outlined in the Declaration of Helsinki. The following is a statement that the experimental process complies with standards and requirements and has been inspected and verified by the unit where it is located in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e\u003cp\u003eA dedicated laboratory setting was prepared to ensure an uninterrupted environment for the participants during their involvement in the study. The lighting conditions in the laboratory were not manipulated and comprised natural light from both indoor fluorescent lamps and outdoor sunlight. To capture facial video data of the participants during their online learning sessions, a computer equipped with a high-definition camera was set up in the laboratory. The participants’ online learning processes were recorded using the EV screen recording software, combined with the high-definition camera.\u003c/p\u003e\u003cp\u003eFor the experimental phase, three approximately 10-minute instructional videos were selected from the Chinese University MOOC website. The videos were titled ‘The Psychology of Love’ ‘Innovative Thinking Behind Open Minds’ and ‘Fundamental Principles of Marxism’. All three online courses were classified as national quality courses offered by the Chinese University MOOC. Corresponding test questions were designed for each course to evaluate the participants’ learning outcomes. The test questions we utilized were carefully selected from the supplementary test materials provided after the MOOC courses. These test questions were evaluated by two experts in the respective field of the course, who confirmed that they accurately reflect students' learning outcomes. The test questions are scored out of 10 and consist of four multiple-choice questions, two fill-in-the-blank questions, and one short-answer question. Participants were required to complete the respective test questions after watching each video to obtain their final test scores. The overall data collection process is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e\u003cp\u003eIn this data collection experiment, a total of 120 segments of online learning videos, each approximately 10 minutes in length, were collected. Building on the research conducted by Whitehill et al., which compared the usefulness of video-based sequences and image-based methods in recognizing engagement levels, this study found that image-based methods had relatively higher accuracy compared to video-based methods. This suggests that engagement is more of a spatial concept rather than a spatiotemporal one.(\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e) Based on these researches, we obtained a total of 71,185 images for further experimentation. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e presents the number and proportion of images associated with each engagement level.\u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eDistribution of the number of images for three levels of emotional engagement.\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"4\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eLearn emotional engagement\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eHighly engaged\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eModerately engaged\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e\u003cp\u003eDisengaged\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eLabel\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e1\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eNumber of pictures\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e16515\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e38468\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e16202\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eThe proportion of the number of pictures\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e23.20%\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003e54.04%\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003e22.76%\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\u003c/div\u003e\u003ch3\u003e2. Research design\u003c/h3\u003e\u003cp\u003eThe research design consists of seven stages to address the objectives and research questions. Here is a detailed description of each stage.\u003c/p\u003e\u003cp\u003eStage 1: Facial data is obtained from online learning environments using a webcam and stored in a database.\u003c/p\u003e\u003cp\u003eStage 2: The collected data undergoes a cleaning process using Camtasia Studio video editing software to remove any data that does not meet the experimental requirements. This step ensures that only valid and relevant data is retained for further analysis. (Camtasia is a software package produced by TechSmith in the United States that integrates computer screen recording and video editing. It also includes built-in features for Camtasia recorder, Camtasia Studio editor, Camtasia menu maker, Camtasia theater, Camtasia player, and Screencast).\u003c/p\u003e\u003cp\u003eStage 3: Expert coders encode the emotion engagement data based on the theory of emotional engagement. The coders carefully analyze and label the collected data with the appropriate emotion engagement categories, applying their expertise and knowledge in emotional engagement research.\u003c/p\u003e\u003cp\u003eStages 4 and 5: These stages involve the exploration of the first research question. The encoded emotion engagement data is utilized to train and evaluate optimized deep learning models. Through various iterations, the models are refined and adjusted to improve their performance in accurately identifying and classifying emotion engagement in the collected facial data.\u003c/p\u003e\u003cp\u003eStage 6: The trained model with the best parameters is employed to identify and assign emotion engagement labels to unlabeled facial data. This allows for the automatic detection and classification of emotion engagement in previously unlabeled data.\u003c/p\u003e\u003cp\u003eStage 7: Statistical analysis methods, such as Pearson correlation analysis, are utilized to address the second and third research questions. The collected data, including the labeled emotion engagement data and associated learning outcomes, are analyzed to examine the relationships between emotion engagement and learning outcomes. Statistical techniques are employed to determine the strength and significance of these relationships.\u003c/p\u003e\u003cp\u003eIn the second stage, the cleaning process involves removing video data that does not meet the experimental requirements. Additionally, the videos are segmented into multiple video segments, ensuring that each segment contains only one category of emotion engagement. Camtasia Studio video editing software is employed for this purpose. Invalid video segments, where the learner’s face is obscured or cannot be detected, are excluded during the segmentation process. Figure\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e illustrates the process of video segmentation using Camtasia Studio software, and Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e displays the results of the video segmentation. In total, 1067 valid video segments were extracted from the initial 120 video segments for analysis in this study. We extracted one frame image every 5 frames from each video segment, excluding images that did not correspond to the engagement level of the video segment. The extracted images were then assigned the engagement level corresponding to their respective video segments. For instance, images extracted from highly engaged video segments were also assigned a highly engaged level. After data cleaning and annotation, a total of 71,185 images were obtained.\u003c/p\u003e\u003ch3\u003e3. Coding scheme\u003c/h3\u003e\u003cp\u003eIn the field of online learning, learners’ facial expressions generate a substantial volume of data, which poses challenges in terms of the time required for manual coding. To overcome this methodological challenge, we have developed an optimized deep learning model. To effectively train and evaluate this model, we have devised an encoding scheme for emotional engagement levels in online learning, drawing upon the theory of learners’ emotional engagement.(\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e)(\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e) The encoding scheme, presented in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, categorizes learners’ emotional engagement into three distinct levels: highly engaged, moderately engaged, and disengaged. Each emotional engagement category is thoroughly described in Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, providing detailed insights into the characteristics and attributes associated with each level of emotional engagement.\u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eCoding scheme for learning emotional engagement in online learning.\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"4\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eClass\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eHead features\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eEye features\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e\u003cp\u003eFacial expression features\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eHighly engaged\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eHead upright or inclined forward\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eStaring at the screen, eyes unconsciously widening, increased distance between upper and lower eyelids\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eSurprise, joy, focus, enthusiasm, and other positive expressions.\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eModerately engaged\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eHead generally upright or slightly tilted to the left or right\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eLine of sight positioned within the screen area, eyes open normally, no change in the distance between upper and lower eyelids\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eCalm, neutral and other neutral expressions\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eDisengaged\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eHead not upright and significant tilt to the left or right\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eThe line of sight is positioned at the edge of the screen area or outside the screen area, eyes slightly closed or even completely closed, and the distance between the upper and lower eyelids decreases\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eBored, tired, indifferent, and other negative expressions\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\u003c/div\u003e\u003cp\u003eIn the third phase, to ensure the quality and credibility of the dataset constructed for learners’ engagement, a crowdsourcing approach was employed for data annotation. Three students with academic backgrounds in educational technology were recruited as data annotators, and they underwent training to familiarize themselves with the relevant definitions of learners’ engagement states, the annotation tools, and the specific definitions of the three engagement labels. During the training, a portion of annotated data was provided for practice, and discussions and Q\u0026amp;A sessions were organized to address any issues or questions encountered by the annotators. Guidance and clarification were provided to resolve doubts or disagreements and to ensure a consensus among the annotators. Based on the performance of the annotators during training, they were confirmed as data annotators to participate in the annotation task.\u003c/p\u003e\u003cp\u003eTo ensure the validity and reliability of the data annotation results, this study adopted a reliability verification method proposed by Kaur et al. A consistency check was conducted using Kendall’s coefficient of agreement.(\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e) The results of Kendall’s coefficient of agreement for the data annotation by all annotators revealed a high level of consistency, with a Kendall’s coefficient of agreement of 0.889 (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:p\u0026lt;0.01\\)\u003c/span\u003e\u003c/span\u003e). This high reliability and accuracy of the data annotation confirm the validity and suitability of the annotated data for training and evaluating the online learning emotion engagement recognition model.\u003c/p\u003e\u003ch3\u003e4. Automatic engagement detection based on Vision Transformer network and transfer learning\u003c/h3\u003e\u003cp\u003eIn the fourth and fifth stages of the study, an analysis of the encoding scheme for the emotional engagement data was conducted, revealing an issue of class imbalance within the collected dataset. Additionally, due to the smaller number of participants and a larger number of training samples per participant, there was limited diversity in the data, leading to a smaller intra-class distance and a larger inter-class distance. To address these challenges and improve the model’s performance, robustness, and generalization capabilities, the study considered the possibility of pretraining the Vision Transformer network model using the DAiSEE dataset.(\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e) The pretrained model’s weights would then be utilized as the initial weights for furt19her training using the self-built emotional engagement dataset. By leveraging the pretrained model and incorporating it into the training process, it was anticipated that the model’s performance and generalization abilities could be enhanced, leading to improved results in recognizing and classifying emotional engagement in the online learning context. This approach aimed to address the issue of limited data diversity and enhance the overall effectiveness of the model.\u003c/p\u003e\u003cp\u003eThe Vision Transformer network model, introduced by Dosovitskiy et al. in 2020, is a notable innovation that adapts the Transformer architecture, originally designed for natural language processing tasks, to the field of computer vision.(\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e) This model represents a self-attention-based approach to image classification. In contrast to traditional convolutional neural networks, the Vision Transformer does not employ convolutional layers but instead relies exclusively on self-attention mechanisms to extract relevant features from images. The architecture of the Vision Transformer model is visualized in Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e, showcasing the arrangement of self-attention layers and feed-forward neural networks. Through the use of self-attention mechanisms, the model captures dependencies between different regions of an image, enabling it to effectively process and understand the visual information. This innovative approach has shown promising results in various computer vision tasks and has the potential to significantly impact the field of image classification.\u003c/p\u003e\u003cp\u003eTo capture more comprehensive and detailed feature information, the Vision Transformer model employs a multi-head self-attention mechanism. This mechanism involves running multiple self-attention mechanisms simultaneously, and then combining their outputs through concatenation and linear transformation to achieve the desired output dimensionality. The calculation formulas for the multi-head self-attention are provided in Equations (\u003cspan refid=\"Equ1\" class=\"InternalRef\"\u003e1\u003c/span\u003e) and (\u003cspan refid=\"Equ2\" class=\"InternalRef\"\u003e2\u003c/span\u003e):\u003c/p\u003e\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$\\:MultiHead\\left(Q,K,V\\right)=Concat\\left({head}_{1},\\cdots\\:,{head}_{h}\\right){W}^{O}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Q\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:K\\)\u003c/span\u003e\u003c/span\u003e, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:V\\)\u003c/span\u003e\u003c/span\u003e represent the query vector matrix, key vector matrix, and value vector matrix, respectively. The MultiHead function concatenates the outputs of each individual self-attention head, denoted as \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{head}_{i}\\)\u003c/span\u003e\u003c/span\u003e, for \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:i=1\\dots\\:h\\)\u003c/span\u003e\u003c/span\u003e (the total number of heads). \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{W}^{O}\\)\u003c/span\u003e\u003c/span\u003e is the weight matrix used for linear transformation. Each self-attention head \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{head}_{i}\\)\u003c/span\u003e\u003c/span\u003e performs the following calculations:\u003c/p\u003e\u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ2\" name=\"EquationSource\"\u003e\n$$\\:{head}_{i}=Attention\\left({Q\\cdot\\:W}_{i}^{Q},{K\\cdot\\:W}_{i}^{K},{V\\cdot\\:W}_{i}^{V}\\right)$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\u003c/div\u003e\u003cp\u003ewhere \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{W}_{i}^{Q}\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{W}_{i}^{K}\\)\u003c/span\u003e\u003c/span\u003e, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{W}_{i}^{V}\\)\u003c/span\u003e\u003c/span\u003e are the learnable weight matrices for the query, key, and value projections of the \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:i\\)\u003c/span\u003e\u003c/span\u003eth self-attention head. The \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:Attention\\)\u003c/span\u003e\u003c/span\u003e function computes the attention scores and applies them to the values to obtain the attended output. During the training process, we first pre-trained the Vision Transformer network model on the DAiSEE dataset, and then fine-tuned it on our self-built dataset of learners’ learning engagement.\u003c/p\u003e\u003ch3\u003e5. Data analysis and automated feedback model\u003c/h3\u003e\u003cp\u003eTo address the first research question, a 10-fold cross-validation method was employed to compare the performance of the Vision Transformer with the baseline model. The parameter settings for the Vision Transformer network model in this study are summarized in Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. These settings were chosen to train the Vision Transformer model effectively and optimize its performance for the task of emotional engagement recognition in online learning.\u003c/p\u003e\u003cdiv class=\"gridtable\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eParameter settings for Vision Transformer network model.\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"3\"\u003e\u003c/colgroup\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eParameters\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eParameter settings\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eDefinition\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eLearning rate\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e0.001\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eLearning rate\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInput image size\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e100×100\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eImage input size of the model\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eOptimizer\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eAdam\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eGradient optimization algorithm\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eepochs\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e500\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eNumber of model iterations\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eBatch size\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e64\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eNumber of samples per batch in one training iteration\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003ePatch size\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003e5×5\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eSize dimensions of image segmentation blocks\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/table\u003e\u003c/div\u003e\u003cp\u003eTo address the second research question, an automatic detection and feedback system was developed. The process flowchart of the system is illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e. This system facilitated the analysis of learner engagement recognition and variations during online learning. The analysis results were communicated to teachers in a timely manner, enabling them to better understand and respond to learners’ engagement levels. In the seventh stage, Pearson correlation analysis was employed to investigate the relationship between learners’ emotional engagement and their learning outcomes, providing insights into the impact of emotional engagement on learning effectiveness.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003e1. Optimized Vision Transformer model identify learners\u0026rsquo; emotional engagement in online learning\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe purpose of this experiment was to evaluate the performance of the optimized Vision Transformer model and transfer learning model in detecting emotional engagement. Firstly, the Vision Transformer network model was pre-trained using the DAiSEE dataset to enhance its feature representation and generalization ability. To assess the impact of transfer learning on model performance, comparative experiments were conducted to evaluate the accuracy of models with and without pre-training. The term \u0026lsquo;without pre-training\u0026rsquo; refers to the Vision Transformer network model trained directly on the self-built learning engagement dataset without prior pre-training on the DAiSEE dataset. The results of the comparative experiments are presented in Table 4.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 4.\u003c/strong\u003e Comparison of experimental results (%) before and after transfer learning.\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\"\u003e\n \u003cp\u003eMacro-Recall\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\"\u003e\n \u003cp\u003eMacro-Precision\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\"\u003e\n \u003cp\u003eMacro-F1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eVision Transformer\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e91.79 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e91.48 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e93.04 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.46\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003e91.99 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.55\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eVision Transformer +transfer learning\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e93.82 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e93.78 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e94.26 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;0.80\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e93.95 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.12\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eFrom Table 4, it can be observed that the model achieves higher recognition accuracy after transfer learning. This improvement can be attributed to the fact that the pre-training dataset (DAiSEE) for the Vision Transformer model also includes learning engagement. The facial features extracted from the DAiSEE dataset are similar to those extracted from the self-built dataset, thereby significantly increasing the quantity and diversity of training data. As a result, the model\u0026rsquo;s accuracy in recognizing learning engagement is enhanced.\u003c/p\u003e\n\u003cp\u003eAdditionally, we compared the classification performance of our proposed optimized Vision Transformer + transfer learning model with other models in the task of emotional engagement detection. The comparison methods were as follows: Gabor+SVM.(42) Decision tree,(41) ResNet+TCN.(26) LDP-KPCA-DBN.(17) The algorithm is described in Section 2.2. Table 5 presents the comparison of the performance results on the self-built dataset between our algorithm and the baseline models.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 5.\u003c/strong\u003e Classification results (%) of emotional engagement.\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eClass\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eGabor+SVM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eDecision Tree\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eResNet+TCN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eLDP-KPCA-DBN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eVisionTransformer +transfer learning\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eHighly engaged\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e77.52 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.61\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e89.22 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e89.93 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e95.52 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.68\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e95.97 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.01\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eModerately engaged\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e73.65 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.82\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e73.17 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.46\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e82.24 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.16\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e85.83 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.41\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e92.46 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;0.79\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eDisengaged\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e82.19 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.81\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e83.15 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e84.02 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.19\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e83.39 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e93.03 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.29\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\"\u003e\n \u003cp\u003eMean\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e77.78 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.52\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e81.84 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e85.40 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.29\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e88.25 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e93.82 \u003cimg width=\"12\" height=\"21\" src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAwAAAAVBAMAAACAghKHAAAAAXNSR0IArs4c6QAAABhQTFRFAAAAAAAAADqQOpDbkDoA25A62//////bpI32zQAAAAF0Uk5TAEDm2GYAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAAZdEVYdFNvZnR3YXJlAE1pY3Jvc29mdCBPZmZpY2V/7TVxAAAAIUlEQVQYV2NgIAGwGoAV46ZKBIFADa8SuHawUXANSK4AANPHA88Im2w2AAAAAElFTkSuQmCC\" alt=\"image\"\u003e\u0026nbsp;1.20\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eFrom Table 5, it can be observed that the pre-training results of the Vision Transformer network model on the DAiSEE dataset outperform the accuracy of the other three baseline model based on the same dataset. Taking the LDP-KPCA-DBN baseline algorithm as an example, our algorithm (Vision Transformer and Vision Transformer + transfer learning) outperforms it by 3.54% and 5.57% respectively. This indicates that the Vision Transformer model exhibits excellent recognition performance in the field of computer vision and performs better in recognizing learning engagement compared to conventional neural networks.\u003c/p\u003e\n\u003ch2\u003e2. Students\u0026rsquo; engagement vary during online learning\u003c/h2\u003e\n\u003cp\u003eFigure 9 illustrates the overall automatic recognition results of learners\u0026rsquo; engagement. The results of the self-built dataset of emotional engagement reveal interesting patterns throughout the online course. It can be observed from the figure that the majority of learners\u0026rsquo; engagement is categorized as \u0026lsquo;moderate engagement\u0026rsquo; throughout the entire duration of the course. However, as the learning session progresses, there is a gradual decline in learners\u0026rsquo; engagement. The proportion of \u0026lsquo;high engagement\u0026rsquo; and \u0026lsquo;moderate engagement\u0026rsquo; decreases, while the proportion of \u0026lsquo;low engagement\u0026rsquo; increases. Notably, around the 6-minute mark, there is a more pronounced increase in the proportion of \u0026lsquo;low engagement\u0026rsquo;, indicating a decline in learners\u0026rsquo; overall engagement during that period. Interestingly, towards the end of the course, around the 8-minute mark, there is a slight improvement in learners\u0026rsquo; engagement, despite the course nearing its conclusion. This suggests a potential rebound or revitalization of engagement towards the end of the learning session. These insights provide essential information about the fluctuations in learners\u0026apos; engagement levels during online learning and highlight specific time periods when learners may experience changes in their level of engagement. Understanding these patterns can be valuable for educators and instructional designers to optimize learning experiences and implement interventions to sustain and enhance learner engagement throughout the course.\u003c/p\u003e\n\u003cp\u003eFigure 10 presents the overall manual annotation results of learners\u0026rsquo; engagement during online learning. The experimental results demonstrate the effectiveness of our proposed online learning sentiment recognition model in accurately identifying and categorizing learners\u0026rsquo; engagement levels. The manual annotation results are generally consistent with the automated detection results, indicating that our algorithm is effective in capturing the overall patterns of learners\u0026rsquo; changes throughout the online learning process. The alignment between manual annotations and automated detection results allows for a deeper analysis of consistent patterns in learners\u0026rsquo; engagement. This highlights the robustness and reliability of our algorithm in providing timely and accurate understanding of learners\u0026rsquo; engagement states. Moreover, the effectiveness of our algorithm becomes particularly evident when comparing it to the automated detection and feedback processes. Our algorithm provides a comprehensive and nuanced understanding of learners\u0026rsquo; engagement, surpassing the limitations of automated detection alone. It enables us to gain valuable insights into the dynamics of learners\u0026rsquo; engagement, facilitating timely and effective interventions to enhance the learning experience. Overall, the experimental results from Figure 10 affirm the effectiveness of our online learning sentiment recognition model in identifying learners\u0026rsquo; engagement during online teaching. This supports the notion that our algorithm is a valuable tool for gaining a deeper understanding of student states and optimizing the online learning environment.\u003c/p\u003e\n\u003ch2\u003e3. The relationship between emotional engagement and learning outcomes?\u003c/h2\u003e\n\u003cp\u003eIn Table 6, the statistical analysis results are presented for the automatic detection results, manual annotations, average of manual annotations, and test results. The metrics included in the table are the mean and standard deviation (SD). For the automatic recognition results, the mean values on three online courses are 2.470,1.931,1.651, respectively, indicating the average emotional engagement level of all learners as determined by the online learning emotion recognition model. The SD represents the variability in the recognition results. The manual annotations by three annotators provide an additional perspective on learners\u0026rsquo; emotional engagement. The average of manual annotations combines the individual annotations from the three annotators, providing a more comprehensive assessment of learners\u0026rsquo; emotional engagement. Lastly, the test scores represent the learners\u0026rsquo; performance on the quiz questions designed to assess their learning effectiveness. The mean and SD values of the test results are provided. These statistical analysis results offer valuable insights into the agreement between the automatic recognition results and manual annotations, as well as the learners\u0026rsquo; learning outcomes. They provide a quantitative assessment of the emotional engagement levels, helping to validate the effectiveness of the online learning emotion recognition model and evaluate the learners\u0026rsquo; understanding and retention of the video content.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 6.\u003c/strong\u003e Statistical analysis of emotional engagement and learning outcomes.\u003c/p\u003e\n\u003cdiv align=\"Left\"\u003e\n \u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"595\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 53px;\"\u003e\n \u003cp\u003eCourse ID\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 63px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 66px;\"\u003e\n \u003cp\u003eAnnotator1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 76px;\"\u003e\n \u003cp\u003eAnnotator2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 66px;\"\u003e\n \u003cp\u003eAnnotator3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003eAverage of manually annotated results\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 82px;\"\u003e\n \u003cp\u003eAutomatic\u003c/p\u003e\n \u003cp\u003edetection\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 51px;\"\u003e\n \u003cp\u003eTest score\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" valign=\"top\" style=\"width: 53px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 63px;\"\u003e\n \u003cp\u003eMean\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e2.782\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e2.224\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e2.565\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e2.523\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 82px;\"\u003e\n \u003cp\u003e2.470\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 51px;\"\u003e\n \u003cp\u003e8.100\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 63px;\"\u003e\n \u003cp\u003eSD\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e0.323\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e0.329\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e0.410\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e0.377\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 82px;\"\u003e\n \u003cp\u003e0.323\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 51px;\"\u003e\n \u003cp\u003e1.370\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 63px;\"\u003e\n \u003cp\u003eN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e120\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 82px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 51px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" valign=\"top\" style=\"width: 53px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 63px;\"\u003e\n \u003cp\u003eMean\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e2.10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1.98\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e1.89\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e1.986\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 82px;\"\u003e\n \u003cp\u003e1.931\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 51px;\"\u003e\n \u003cp\u003e6.300\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 63px;\"\u003e\n \u003cp\u003eSD\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e0.201\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e0.216\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e0.260\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e0.226\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 82px;\"\u003e\n \u003cp\u003e0.245\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 51px;\"\u003e\n \u003cp\u003e0.823\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 63px;\"\u003e\n \u003cp\u003eN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e120\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 82px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 51px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" valign=\"top\" style=\"width: 53px;\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 63px;\"\u003e\n \u003cp\u003eMean\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e1.73\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e1.80\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e1.63\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e1.716\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 82px;\"\u003e\n \u003cp\u003e1.651\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 51px;\"\u003e\n \u003cp\u003e4.400\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 63px;\"\u003e\n \u003cp\u003eSD\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e0.122\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e0.174\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e0.158\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e0.175\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 82px;\"\u003e\n \u003cp\u003e0.121\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 51px;\"\u003e\n \u003cp\u003e0.516\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 63px;\"\u003e\n \u003cp\u003eN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 76px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 66px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e120\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 82px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 16px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 51px;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003eTable 7 presents the results of the Pearson correlation analysis conducted to examine the relationship between learners\u0026rsquo; emotional engagement and their learning outcomes. The variables analyzed include emotional engagement through manual annotation and automatic detection, as well as learning outcomes. Whether it is automatic emotion recognition or manual annotation results, there is a significant relationship with the learning outcomes. Specifically, the coefficient correlations between automatic detection and test score were \u003cem\u003er\u003c/em\u003e=0.860**,0.664*, and 0.707*. The coefficient correlations between manual annotation and test score were \u003cem\u003er\u003c/em\u003e=0.799**,0.657*, and 0.636*. This suggests that the correlation between emotional engagement and learning outcomes is statistically significant. The results of the Pearson correlation analysis support the hypothesis that learners\u0026rsquo; emotional engagement has a positive influence on their learning outcomes. This experimental result emphasizes the importance of emotional engagement in online learning and suggests that learners who are more emotionally engaged tend to achieve better learning outcomes. Overall, the results of the Pearson correlation analysis provide evidence of a significant and positive correlation between learners\u0026rsquo; emotional engagement and their learning outcomes, highlighting the relevance of emotional engagement in the context of online learning.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 7.\u003c/strong\u003e The Pearson correlation analysis between test score, automatic detection and manual annotation.\u003c/p\u003e\n\u003cdiv align=\"Left\"\u003e\n \u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 64px;\"\u003e\n \u003cp\u003eCourse ID\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 163px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 111px;\"\u003e\n \u003cp\u003eManual annotation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 94px;\"\u003e\n \u003cp\u003eAutomatic detection\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003eTest score\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" style=\"width: 64px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 163px;\"\u003e\n \u003cp\u003eManual annotation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 111px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 94px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 163px;\"\u003e\n \u003cp\u003eAutomatic detection\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 111px;\"\u003e\n \u003cp\u003e0.869**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 94px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 163px;\"\u003e\n \u003cp\u003eTest score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 111px;\"\u003e\n \u003cp\u003e0.799**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 94px;\"\u003e\n \u003cp\u003e0.860**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" style=\"width: 64px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 163px;\"\u003e\n \u003cp\u003eManual annotation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 111px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 94px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 163px;\"\u003e\n \u003cp\u003eAutomatic detection\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 111px;\"\u003e\n \u003cp\u003e0.876**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 94px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 163px;\"\u003e\n \u003cp\u003eTest score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 111px;\"\u003e\n \u003cp\u003e0.657*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 94px;\"\u003e\n \u003cp\u003e0.664*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"3\" style=\"width: 64px;\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 163px;\"\u003e\n \u003cp\u003eManual annotation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 111px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 94px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 163px;\"\u003e\n \u003cp\u003eAutomatic detection\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 111px;\"\u003e\n \u003cp\u003e0.887**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 94px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 163px;\"\u003e\n \u003cp\u003eTest score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 111px;\"\u003e\n \u003cp\u003e0.636*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 94px;\"\u003e\n \u003cp\u003e0.707*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 123px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\" valign=\"top\" style=\"width: 337px;\"\u003e\n \u003cp\u003eNote. N=40, *\u003cem\u003ep\u003c/em\u003e\u0026lt;0.05, **\u003cem\u003ep\u003c/em\u003e\u0026lt;0.01, **\u003cem\u003ep\u003c/em\u003e\u0026lt;0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 94px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn this study, we conducted an analysis of the emotional engagement of 40 college students in e-learning courses. By developing emotional engagement detection and feedback models, we examined the dynamics of students\u0026rsquo; emotional engagement throughout the online learning process and investigated whether there were significant variations in emotional engagement across the course. Additionally, we explored the relationship between changes in emotional engagement and academic achievement.\u003c/p\u003e\u003cp\u003eTo address the challenges of limited sample diversity and sample imbalance in facial analysis during online learning, we developed an optimized emotional engagement detection model using the Vision Transformer architecture and transfer learning techniques. From the results presented in Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e and Table\u0026nbsp;\u003cspan refid=\"Tab5\" class=\"InternalRef\"\u003e5\u003c/span\u003e, it is evident that our proposed emotional engagement detection model outperformed existing models in terms of classification accuracy. Compared to traditional machine methods and shallow neural networks that require manual feature design, our model achieved significant improvements in performance. Specifically, our model demonstrated an improvement of 3.54% and 5.57% in classification accuracy compared to existing emotional engagement detection models. This improvement highlights the effectiveness of our approach, which utilizes Vision Transformer and transfer learning techniques to capture richer contextual information and enhance the model\u0026rsquo;s generalization capabilities.\u003c/p\u003e\u003cp\u003eThe analysis of learners\u0026rsquo; emotional engagement revealed interesting patterns. Figure\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e highlighted that learners generally exhibited a state of moderate engagement in the initial minutes of the course. However, as time progressed, their engagement gradually declined, with a sharp drop around the six-minute mark, leading to disengagement in some learners. This research result emphasizes the importance of capturing learners\u0026rsquo; attention and motivation early in the learning process. Educators should incorporate engaging activities and content at the beginning to foster enthusiasm and enhance learners\u0026rsquo; motivation.(\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e) Notably, towards the end of the course, there was a slight recovery in learners\u0026rsquo; emotional engagement, potentially due to the awareness that the course was nearing its conclusion. This observation suggests that learners may become more engaged when they realize the final stage of the course is approaching.\u003c/p\u003e\u003cp\u003eThe statistical analysis results presented in Table\u0026nbsp;\u003cspan refid=\"Tab6\" class=\"InternalRef\"\u003e6\u003c/span\u003e and Table\u0026nbsp;\u003cspan refid=\"Tab8\" class=\"InternalRef\"\u003e7\u003c/span\u003e demonstrated a positive correlation between learners\u0026rsquo; emotional engagement and their learning effectiveness. Both the automated recognition results and manual annotations indicated that learners\u0026rsquo; engagement positively influenced their learning outcomes. When learners were interested and fully engaged, their learning effectiveness tended to be higher. On the other hand, if learners experienced boredom or lacked full engagement, their learning effectiveness was negatively impacted. These research results highlight the importance of monitoring and addressing learners\u0026rsquo; emotional engagement during teaching. Teachers should strive to introduce stimulating content and activities to enhance learners\u0026rsquo; engagement and improve learning effectiveness. The positive correlation between emotional engagement and learning effectiveness aligns with the understanding that emotions play a crucial role in driving learning outcomes. Emotions play a significant role in learning achievement. This research result is consistent with previous research emphasizing the impact of emotions on education.(\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e)\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003eGiven the paramount importance of the interaction between emotional engagement and learning outcomes, this study employs an optimized deep learning model and employed statistical analysis methods to explore the complex relationships between them. We collected facial data from 40 college students during their online learning to come up with a dataset containing 71,185 instances of emotional engagement. On this basis, an automatic detection and feedback model for emotional engagement is constructed. The results of this study are twofold. First, compared to existing research in the field, the developed model shows a significant improvement in recognition performance. This marks a significant breakthrough in accurately identifying and interpreting emotional engagement in online learning environments. Second, Pearson correlation analysis revealed a significant relationship between emotional engagement and learning outcomes. Specifically, those learners who demonstrate genuine interest and total commitment to the learning material generally achieve higher academic achievement. Conversely, individuals who experience apathy or struggle to fully engage in the learning process often show impaired learning outcomes.\u003c/p\u003e\u003cp\u003eBased on these studies, several suggestions to foster student motivation and optimize learning outcomes can be offered. Firstly, educators can curate captivating and intellectually stimulating discussion tasks to engender a comprehensive emotional engagement among learners. This approach fosters deep cognitive involvement and enhances the overall learning experience.(\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e)(\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e) Secondly, the machine learning-based automated learning intervention techniques has shown great promise in bolstering online learning outcomes.(\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e) By incorporating automated components within online learning platforms, educators can dynamically adjust and optimize the level of emotional engagement among learners. Finally, it is recommended that the broad potential of mood monitoring devices be fully utilized in educational institutions. These devices can provide valuable insights into students\u0026rsquo; emotions, helping teachers gather feedback and adjust teaching methods accordingly.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003eFunding\u003c/p\u003e\n\u003cp\u003eFunding: National Natural Science Foundation of China (No. 62177032), “Research on the Autonomous Training and Evaluation Model for Pre-service Teachers’ Classroom Teaching Expression Competence.”\u003c/p\u003e\n\u003cp\u003eAuthor contributions statement\u003c/p\u003e\n\u003cp\u003eGuanyu Chen is responsible for article writing and revision, as well as communication work.\u003c/p\u003e\n\u003cp\u003eGuangxin Han is responsible for data calculation and processing, as well as icon creation and modification.\u003c/p\u003e\n\u003cp\u003eJuan Niu has provided academic research and theoretical support for this study.\u003c/p\u003e\n\u003cp\u003eJuhou He is the administrator of this project and provides guidance throughout the research process.\u003c/p\u003e\n\u003cp\u003eData availability statement\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe data involved in this study are public. If you need them, please contact the author by email. The author will provide the complete data and related information.\u003c/p\u003e\n\u003cp\u003eAdditional information\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests:\u0026nbsp;\u003c/strong\u003eThe authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eLudjon Roshi, E-learning Statistics 2023:Online Learning Facts[EB/OL]. https://codeless.co/elearning-statistics/, 2023.\u003c/li\u003e\n \u003cli\u003eGemma Josep. 5 reasons why Online learning is the future of Education in 2023[EB/OL].https://www.educations.com/articles-and-advice/5-reasons-online-learning-is-future-of-education-17146, 2023.\u003c/li\u003e\n \u003cli\u003eMeans, Barbara and Toyama, Yuki and Murphy, Robert and Bakia, Marianne and Jones, Karla. Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies. Project Report. Centre for Learning Technology[M]. Association for learning technology, 2009(8).\u003c/li\u003e\n \u003cli\u003eJoshua Gans. What is the future of online education?[EB/OL]. https://www.weforum.org/agenda/2014/12/what-is-the-future-of-online-education/, 2014.\u003c/li\u003e\n \u003cli\u003e中国互联网络信息中心. 第50次中国互联网网络发展状况统计报告[EB/OL]. https://www.cnnic.net.cn/n4/2022/0914/c88-10226.html, 2022-08-31.\u003c/li\u003e\n \u003cli\u003e祝智庭, 胡姣. 技术赋能后疫情教育创变: 线上线下融合教学新样态[J]. 开放教育研究, 2021, 27(1): 13-23.\u003c/li\u003e\n \u003cli\u003e刘司卓,李爽,黄嘉靖.直播课学习行为投入评价的实证研究[J].中国远程教育,2021(2):36-45,58.\u003c/li\u003e\n \u003cli\u003eVenton B J, Pompano R R. Strategies for enhancing remote student engagement through active learning[J]. Analytical and Bioanalytical Chemistry, 2021, 413(6): 1507-1512.\u003c/li\u003e\n \u003cli\u003eS\u0026uuml;mer \u0026Ouml;, Goldberg P, D\u0026apos;Mello S, et al. Multimodal engagement analysis from facial videos in the classroom[J]. IEEE Transactions on Affective Computing, 2021.\u003c/li\u003e\n \u003cli\u003e魏艳涛, 雷芬, 胡美佳, 等. 学生表情识别研究综述[J]. 中国教育信息化, 2020(21):48-55.\u003c/li\u003e\n \u003cli\u003eBaker R S, D\u0026apos;Mello S K, Rodrigo M M T, et al. Better to be frustrated than bored: The incidence, persistence, and impact of learners\u0026rsquo; cognitive\u0026ndash;affective states during interactions with three different computer-based learning environments[J]. International Journal of Human-Computer Studies, 2010, 68(4): 223-241.\u003c/li\u003e\n \u003cli\u003eD\u0026rsquo;Mello S, Lehman B, Pekrun R, et al. Confusion can be beneficial for learning[J]. Learning and Instruction, 2014, 29: 153-170.\u003c/li\u003e\n \u003cli\u003eJagers R J, Rivas-Drake D, Williams B. Transformative social and emotional learning (SEL): Toward SEL in service of educational equity and excellence[J]. Educational Psychologist, 2019, 54(3): 162-184.\u003c/li\u003e\n \u003cli\u003ePabba C, Kumar P. An intelligent system for monitoring students\u0026apos; engagement in large classroom teaching through facial expression recognition[J]. Expert Systems, 2022, 39(1):e12839.1-e12839.28.\u003c/li\u003e\n \u003cli\u003e曹晓明, 张永和, 潘萌, 等. 人工智能视域下的学习参与度识别方法研究\u0026mdash;\u0026mdash;基于一项多模态数据融合的深度学习实验分析[J]. 远程教育杂志, 2019, 37(1): 32-44.\u003c/li\u003e\n \u003cli\u003eBosch N, Chen Y, D\u0026rsquo;Mello S. It\u0026rsquo;s written on your face: detecting affective states from facial expressions while learning computer programming[C]. Intelligent Tutoring Systems: 12th International Conference, ITS 2014, Honolulu, HI, USA, June 5-9, 2014 Proceedings 12. Springer, 2014:39-44.\u003c/li\u003e\n \u003cli\u003eDewan M, Murshed M, Lin F. Engagement detection in online learning: a review[J]. Smart Learning Environments, 2019, 6(1): 1-20.\u003c/li\u003e\n \u003cli\u003e张琪, 武法提. 学习分析中的生物数据表征\u0026mdash;\u0026mdash;眼动与多模态技术应用前瞻[J]. 电化教育研究, 2016, 37(9): 76-81.\u003c/li\u003e\n \u003cli\u003eD\u0026apos;Mello S K, Craig S D, Graesser A C. Multimethod assessment of affective experience and expression during deep learning[J]. International Journal of Learning Technology, 2009, 4(3-4): 165-187.\u003c/li\u003e\n \u003cli\u003eD\u0026rsquo;Mello S K, Graesser A. Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features[J]. User Modeling and User-Adapted Interaction, 2010, 20: 147-187.\u003c/li\u003e\n \u003cli\u003eWhitehill J, Serpell Z, Lin Y-C, et al. The faces of engagement: Automatic recognition of student engagementfrom facial expressions[J]. IEEE Transactions on Affective Computing, 2014, 5(1): 86-98.\u003c/li\u003e\n \u003cli\u003eKamath A, Biswas A, Balasubramanian V. A crowdsourced approach to student engagement recognition in e-learning environments[C]. 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2016:1-9.\u003c/li\u003e\n \u003cli\u003eGupta A, D\u0026apos;Cunha A, Awasthi K, et al. Daisee: Towards user engagement recognition in the wild[J]. arXiv preprint arXiv:160901885, 2016.\u003c/li\u003e\n \u003cli\u003eKaur A, Mustafa A, Mehta L, et al. Prediction and localization of student engagement in the wild[C]. 2018 Digital Image Computing: Techniques and Applications (DICTA). IEEE, 2018:1-8.\u003c/li\u003e\n \u003cli\u003eMukhopadhyay M, Pal S, Nayyar A, et al. Facial emotion detection to assess Learner\u0026apos;s State of mind in an online learning system[C]. Proceedings of the 2020 5th international conference on intelligent information technology. 2020:107-115.\u003c/li\u003e\n \u003cli\u003eAbedi A, Khan S S. Improving state-of-the-art in detecting student engagement with resnet and tcn hybrid network[C]. 2021 18th Conference on Robots and Vision (CRV). IEEE, 2021:151-157.\u003c/li\u003e\n \u003cli\u003e王嘉豪, 徐敏, 孙众, 等. 数据分布不平衡的课堂参与度自动识别研究[J]. 小型微型计算机系统: 1-8.\u003c/li\u003e\n \u003cli\u003eWei Q, Sun B, He J, et al. BNU-LSVED 2.0: Spontaneous multimodal student affect database with multi-dimensional labels[J]. Signal Processing: Image Communication, 2017, 59: 168-181.\u003c/li\u003e\n \u003cli\u003eBian C, Zhang Y, Yang F, et al. Spontaneous facial expression database for academic emotion inference in online learning[J]. IET Computer Vision, 2019, 13(3): 329-337.\u003c/li\u003e\n \u003cli\u003eSchmieder A. A glossary of educational reform[J]. Journal of Teacher Education, 1973, 24(1): 55-62.\u003c/li\u003e\n \u003cli\u003eFisher C W, Berliner D C, Filby N N, et al. Teaching behaviors, academic learning time, and student achievement: An overview[J]. The Journal of classroom interaction, 1981, 17(1): 2-15.\u003c/li\u003e\n \u003cli\u003eSkinner E A, Belmont M J. Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year[J]. Journal of educational psychology, 1993, 85(4): 571-581.\u003c/li\u003e\n \u003cli\u003eConnell J P, Wellborn J G. Competence, Autonomy, and Relatedness: A motivational analysis of self-system processes[J]. Journal of Personality and Social Psychology, 1991(65):43-77.\u003c/li\u003e\n \u003cli\u003eFredricks J A, Blumenfeld P C, Paris A H. School engagement: Potential of the concept, state of the evidence[J]. Review of educational research, 2004, 74(1): 59-109.\u003c/li\u003e\n \u003cli\u003ePekrun R, Lichtenfeld S, Marsh H W, et al. Achievement emotions and academic performance: Longitudinal models of reciprocal effects[J]. Child development, 2017, 88(5): 1653-1670.\u003c/li\u003e\n \u003cli\u003eKahu E R, Nelson K. Student engagement in the educational interface: Understanding the mechanisms of student success[J]. Higher education research \u0026amp; development, 2018, 37(1): 58-71.\u003c/li\u003e\n \u003cli\u003eAlyuz N, Okur E, Oktay E, et al. Towards an emotional engagement model: Can affective states of a learner be automatically detected in a 1:1 learning scenario?[C]. 24th ACM Conference on User Modeling, Adaptation and Personalization (UMAP). 2016.\u003c/li\u003e\n \u003cli\u003eAslan S, Mete S E, Okur E, et al. Human expert labeling process (HELP): towards a reliable higher-order user state labeling process and tool to assess student engagement[J]. Educational Technology, 2017: 53-59.\u003c/li\u003e\n \u003cli\u003eSidney K D, Craig S D, Gholson B, et al. Integrating affect sensors in an intelligent tutoring system[C]. Affective Interactions: The Computer in the Affective Loop Workshop at. 2005:7-13.\u003c/li\u003e\n \u003cli\u003e李振华. 融入多源数据的在线学习投入测评方法研究[D]. 华中师范大学, 2020.\u003c/li\u003e\n \u003cli\u003eKuh G D . The national survey of student engagement: Conceptual and empirical foundations[J]. New Directions for Institutional Research, 2010, 2009(141):5-20.\u003c/li\u003e\n \u003cli\u003eWhitehill J, Serpell Z, Lin Y C, et al. The faces of engagement: Automatic recognition of student engagement from facial expressions[J]. IEEE Transactions on Affective Computing, 2014,(1):86-98.\u003cstrong\u003e\u003cbr\u003e\u003c/strong\u003e\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Emotional Engagement, Online Learning, Artificial Intelligence in Education, Educational Data Mining","lastPublishedDoi":"10.21203/rs.3.rs-7581467/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7581467/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eOnline education offers flexibility but often suffers from reduced learner engagement. This study developed an automated method to detect emotional engagement using an optimized Vision Transformer model with transfer learning. Facial data from 40 undergraduates produced a dataset of 71,185 labeled images across three engagement levels. The proposed model achieved 93.8% classification accuracy, surpassing conventional machine learning and deep learning baselines. Analysis showed engagement typically declined after six minutes of learning, with a modest rebound near session end. Pearson correlation revealed a significant positive relationship between engagement and learning outcomes, indicating that emotionally engaged learners achieved higher academic performance. These results demonstrate the feasibility of deep learning\u0026ndash;based approaches for scalable monitoring of learner engagement and highlight the central role of emotional states in shaping online learning effectiveness. The findings provide practical insights for designing adaptive interventions to sustain attention and optimize digital learning environments.\u003c/p\u003e","manuscriptTitle":"Understanding the Impact of Emotional Engagement on Learning Outcomes in Online Education: An Automated Analysis Approach","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-21 23:47:23","doi":"10.21203/rs.3.rs-7581467/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-11-07T11:07:47+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-06T03:00:04+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-10-30T14:08:25+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"183201582931279117202889177721668349450","date":"2025-10-30T07:45:23+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"129693827198830484685976168099012538462","date":"2025-10-30T01:15:01+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-10-29T10:38:10+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-10-22T08:47:23+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-09-19T09:20:35+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-09-19T08:48:42+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2025-09-19T08:42:40+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"259881f7-13cd-4520-a4f2-86c40f9c8802","owner":[],"postedDate":"October 21st, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":56597142,"name":"Physical sciences/Mathematics and computing"},{"id":56597143,"name":"Biological sciences/Psychology"},{"id":56597144,"name":"Social science/Psychology"}],"tags":[],"updatedAt":"2026-01-12T16:07:32+00:00","versionOfRecord":{"articleIdentity":"rs-7581467","link":"https://doi.org/10.1038/s41598-025-34871-x","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2026-01-07 15:57:59","publishedOnDateReadable":"January 7th, 2026"},"versionCreatedAt":"2025-10-21 23:47:23","video":"","vorDoi":"10.1038/s41598-025-34871-x","vorDoiUrl":"https://doi.org/10.1038/s41598-025-34871-x","workflowStages":[]},"version":"v1","identity":"rs-7581467","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7581467","identity":"rs-7581467","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}