An Adaptive Smart Architecture Using Real-Time Multimodal Learning Analytics and Deep Reinforcement Learning

An Adaptive Smart Architecture Using Real-Time Multimodal Learning Analytics and Deep Reinforcement Learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article An Adaptive Smart Architecture Using Real-Time Multimodal Learning Analytics and Deep Reinforcement Learning Jiangbo Li¹, Jun Cheng², Mingming Li³ This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9117316/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract Personalised learning at scale is one of the core challenges in intelligent tutoring systems (ITS) mainly because learners states are high dimensional, learning modalities are heterogeneous, and learner behaviour changes over time. The article presents the Adaptive Smart Architecture of Real-Time Multimodal Learning (ASARTML), a new framework combines real-time multimodal learning analytics and deep reinforcement learning (DRL) engine to provide dynamically responsive and context-sensitive pedagogical content. ASARTML combines six data streams, including clickstream logs, eye-tracking cues, electroencephalography (EEG), scoring on tests, facial action unit in videos, and natural language interaction based on a cross-attention multimodal fusion network. The policy backbone is a dueling deep Q-network with a long short-term memory (LSTM) module. The architecture also includes a Bayesian reward model, which integrates instant feedback of performance and long-term alignment of learning through the knowledge graph. Empirical analysis of 1,240 participants shows that ASARTML gives a classification accuracy of 87.3% in identifying state of learners (F1 = 0.869), statistically significant average learning of 34.8% in comparison with a static curriculum control (Cohen d = 1.84, p = 0.0001). These findings highlight why multimodal DRL systems can significantly expand adaptive learning on an institutional level. Clinical Trial Registration: ClinicalTrials.gov, identifier NCT06284721 ; registered 15 February 2024. Biological sciences/Computational biology and bioinformatics Physical sciences/Mathematics and computing Biological sciences/Neuroscience adaptive learning systems deep reinforcement learning multimodal learning analytics intelligent tutoring real-time adaptation attention fusion Figures Figure 1 Figure 2 Figure 3 1. Introduction The increased development of online and blended learning platforms around the world has produced volumes of interaction with learners never before, but the overwhelming majority of implemented educational systems are still delivering information using unresponsive, one-size-fits-all course materials. It is this structural incompatibility between the provision of learner data and the adaptability of learning systems that is the immediate bottleneck to large scale personalized education. Intelligent tutoring systems (ITS) are not a recent idea, but tracing back to the pioneering efforts of Carbonell (1970) and the fundamental Bloom (1984) study which established a two-standard deviations above classroom performance increase as evidence of the so-called 2-sigma problem. Although decades have passed, modern ITS systems seldom even aspire to this standard, in part due to the fact that they often depend on manual pedagogical heuristics, the use of unidimensional learner models or other simplistic feedback systems that do not reflect the multidimensional, temporally dynamic nature of human cognition. Recent innovations in deep learning, reinforcement learning and multimodal signal processing have provided new opportunities to overcome these limitations. The deep reinforcement learning (DRL) is especially attractive to adaptive tutoring since the process of choosing instructional actions can be naturally represented as a Markov decision process (MDP): an agent monitors a state of a learner, takes an instructional action, obtains a reward signal based on learning results, and enhances its policy. Previous studies by Rafferty et al. (2016) and Chi et al. (2011) showed that relatively simple RL-based tutors were able to outperform human decision-making even under controlled conditions. They were, however, formulations based on highly pre-processed, low-dimensional state representations and did not scale to operate in real time in diverse data streams. At the same time, it has been shown that behavioral, physiological, and linguistic cues significantly can build far richer and more powerful models of learner cognitive and affective conditions than individual modalities (Blikstein and Worsley, 2016; Ochoa, 2017). Physiological correlates including EEG-generated engagement measures and pupil dilation due to eye-tracking have been demonstrated to correlate with cognitive load, attention and emotional valence -constructs that mediate directly on learning effectiveness. The conceptual misconceptions and the metacognitive strategies that are not seen in the clickstream data can be identified by the natural language processing of learner discourse. Although these results are available, no previous system has been able to combine a complete multimodal learning analytics pipeline with a DRL-based adaptive engine, though under real-world latency requirements. The paper bridges this gap by introducing ASARTML (Adaptive Smart Architecture for Real-Time Multimodal Learning): a converged end-to-end learning system that: (i) continuously feeds on six heterogeneous data streams at sub-second latency; (ii) codes these streams into a single common representation in the form of a latent learner state by a cross-attention transformer fusion network; (iii) uses a dueling DQN-LSTM policy to pick the most desirable pedagogical interventions as an action in a structured action space; and (iv) employs a Bayesian reward model grounded in a knowledge graph to balance short-term engagement with long-term knowledge acquisition. The main contributions of the work are as follows: A new, cross-attention-based multimodal learning analytics pipeline with a 6-data-modal and below latency. A DRL policy architecture with dueling DQN and LSTM temporal modeling, which allows learning long learning period policies. A Knowledge graph-informed Bayesian reward model combining mastery alignment with instant learner performance indicators. A randomized controlled trial (n = 1,240) in subjects range of three domains revealing statistically significant changes of three improvement systems, compared to four baseline systems. A full ablation experiment that measures the marginal utility of every architectural element and data modality. 2. Related Work 2.1 Intelligent Tutoring Systems and Adaptive Learning The intelligent tutoring systems have gone through a number of architectural generations. Simple systems like GUIDON (Clancey, 1982) and Lisp-Tutor (Anderson et al., 1985) used model- and knowledge-tracing algorithms in order to deduce what students were or were not learning and give them specific feedback. Bayesian Knowledge Tracing (BKT) developed by Corbett and Anderson (1994) offered a probabilistic model of the learning process of skills as time progressed. More recent deep knowledge tracing (DKT) models, proposed by Piech et al. (2015), scaled-up these models with recurrent neural network models, where long-range temporal dependencies can be captured in the sequence of interactions between learners with significantly higher predictive performance on large-scale learning tasks. The pseudonymous learning management systems (LMSs) and massive open online courses (MOOCs) created a new need on scalable personalization. Learning content recommendation was done using collaborative filtering and matrix factorization methods that had been developed based on recommender systems. Nevertheless, these methods optimize on engagement measures instead of learning goals that are based on pedagogical foundations and do not have state-aware decision making mechanisms in real time. Wei et al. (2021) have shown that even advanced recommendation strategies may not be able to consider affect and motivation of learners, creating high-technical results, but suboptimal pedagogically content sequences. 2.2 Reinforcement Learning in Educational Contexts Rafferty et al. (2016) formalized the application of reinforcement learning to the adaptive tutoring domain and modeled the tutoring task as a partially observed MDP and demonstrated that reinforcement learning agents could perform better than random and myopic decision policies in experimental settings. Later studies by Rowe et al. (2010) on Crystal Island and Mandel et al. (2014) showed RL-controlled narration bifurcation and lecture advice, respectively. Deep Q-networks (DQN) were introduced by Mnih et al. (2015), thus increasing the amount of knowledge that can be represented by RL-based tutors; Liu et al. (2019) used DQN to select hints in an algebra tutoring system and found a 12.4 percent improvement in knowledge acquisition compared to rule-based control groups. Recent research by Doroudi et al. (2019) and Chahoud et al. (2025) has identified some of the fundamental issues with RL application to education, such as sparse and delayed rewards, potential optimization of short-term engagement at the expense of long-term retention, sample inefficiency, and non-stationarity in the state of learners. Partial mitigations have been suggested such as reward shaping techniques, curriculum learning and Bayesian optimization and no consensus architecture has been established. Importantly, the existing body of previous DSR-based tutoring systems has been built almost entirely on performance measures (assessment scores, response times) as state observations without taking the more meaningful learner signal provided by physiological and behavioral modalities. 2.3 Multimodal Learning Analytics According to Blikstein and Worsley (2016), the concept of multimodal learning analytics includes the analysis of concurrent data streams to comprehend and maximize learning processes through automated methods. Basic research provided arguments of predictive validity of individual modalities: D'Mello and Graesser (2012) created evidence that facial action coding was correlated with states of confusion and frustration among learners; Amadieu et al. (2009) provided evidence that fixation patterns during eye-tracking corresponded with the understanding of the text-document navigation; and Crowley et al. (2010) demonstrated that the frontal theta activity of the EEG reflected the working memory load and learning performance. The issues of fusing these modalities which are different in sampling rate, dimensionality, noise properties, semantic granularity have been tackled using early, late and hybrid fusion architecture. Neural architectures that are attention-based have had a specific success in multimodal fusion. The cross-modal attention structure provided by Lu et al. (2019) in ViLBERT, which was originally a vision-language task, has been expanded into educational multimodal fusion by a number of groups (Manzoor et al., 2025; Dewan et al., 2018). The methods acquire weighted functions of associations between features of the modality in order to allow the model to pick out the most informative features to a particular learner scenario. Although these improvements are provided, there is little literature on the integration of real-time MMLA and DRL-based adaptive engines, which is the main gap that this work will fill. 2.4 Explainability in Educational AI With the increase in AI-based educational technologies, the issues of algorithmic responsibility, educator agency, and student trustfulness have become heated (Selwyn, 2019; Luckin and Holmes, 2022). Post-hoc explanation Post-hoc explanation has been applied to student outcome prediction (Conijn et al., 2023) and exercise recommendation systems (Raza et al., 2022) using explainable AI (XAI) methods such as the LIME (Ribeiro et al., 2016), SHAP (Lundberg and Lee, 2017), counterfactual explanations (Wachter et al., 2017), and concept activation vectors (Kim et al., 2018). But, to produce the reasons behind sequential policy decisions, in which the justification of a given suggestion is based on a convoluted course of past states and actions, is difficult in a way that single-prediction explanation strategies are not meant to handle. The first principled solution to this issue in the educational AI setting is our XPM module, a combination of trajectory-level SHAP attribution as well as counterfactual scenario simulation and template-based NLG. 3. Materials and Methods 3.1 System Overview ASARTML is an architecture that integrates five layers into a modular and real-time adaptive framework known as: (1) Multimodal Data Acquisition Layer, which is in charge of collecting and pre-processing heterogeneous sensor and behavioral layers synchronously (ASARTML). (2) Feature Extraction and Temporal Alignment Layer, which converts raw signals into standardized feature representations (ASARTML). (3) Cross-Attention Fusion Network, which learns to generate a single latent learner state vector (ASARTML). (4) DRL Figure 1 gives a schematic description of the system architecture. The full inference process, including sensor data consumption and intervention application, is expected to be able to run within 2 seconds of latency. Python 3.10 with PyTorch 2.1 as the deep learning components, a Ray RLlib framework to train distributed RL, Apache Kafka to stream real-time data and MongoDB time-series database to store learner session history was used to implement this system. The LMS interface has been developed based on the Open edX platform extended. The entire training of DRA was performed on a cluster of 8 × NVIDIA A100 (80GB) GPUs. Neo4j was used to build the knowledge graph which was populated with concept prerequisites and difficulty metadata (based on curricula expert annotation and automated analysis of 4.2 million records of learner interactions in a previous deployment). 3.2 Multimodal Data Collection The data came as a form of a learning laboratory with Tobii Pro Spectrum eye-trackers (120 Hz), EMOTIV EPOC X 14-channel EEG headsets (256 Hz), high-definition webcams (1080p, 30 fps) to detect facial action units, and regular work stations on which the ASARTML-instrumented LMS client was running. There was also interaction with an inbuilt natural language chat tutor. Every hardware was connected through a network time protocol (NTP) master clock at less than 5 ms jitter. Table 1 summarizes the datasets involved in the training and evaluation of the system, and includes the multimodal data sources, the type of modality of each, sample, features, and the sampling rates. Table 1. Summary of multimodal data sources used in ASARTML training and evaluation. Data Source Modality Samples (n) Features Sampling Rate Clickstream Log Behavioral 142,800 87 event types Real-time (1 s) Eye-Tracking Physiological 38,400 Fixation, saccade, pupil diam. 120 Hz EEG (14-channel) Neurophysiol. 12,960 Alpha, beta, theta bands 256 Hz Assessment Scores Cognitive 28,500 Pre/post quiz, rubric Per-module Interaction Video Visual 9,620 sessions Facial AUs, head pose 30 fps Natural Language (chat) Linguistic 89,300 utterances Sentiment, intent, entities Event-driven There were modality-specific preprocessing pipelines. I-VT fixation classification with a velocity threshold of 30°/s was used to process eye-tracking data. The EEG signals were filtered using band-pass (1-45 Hz) and artifact-rejected using independent component analysis (ICA) and decomposed into the alpha (8-13 Hz), beta (14-30 Hz) and theta (4-7 Hz) power bands using short-time fourier transform with a 512 sample window and 50 percent overlap. Video data were manipulated with the help of MediaPipe Face Mesh model that identifies 468 facial landmarks, which were used to calculate 17 intensity of AUs. The inputs were converted to natural language and were tokenized and encoded with a fine-tuned RoBERTa-base model. The events were modeled as one-hot encoded event-type vectors with session relative timestamps and dwell-time characteristics as clickstream events. 3.3 Feature Extraction and Temporal Alignment Since the sampling rates of the different modalities were heterogeneous (between event-driven NLP input and 256 Hz EEG), a temporal alignment protocol was introduced. All modality-specific feature extractors generated normalized embedding vectors of d = 128 at a single output cadence of 1 Hz by sliding-window aggregation. EEG and eye-tracking features were summed in 1-second windows; video features were summed in 30-frame windows; NLP features were summed as count-weighted histograms in 1-second windows; and clickstream features were summed as count-weighted histograms in 1-second windows. Such unification allows the cross-attention fusion network to be performed on a timed grid. To apply the EEG stream specifically, a calibration period of 5 minutes was performed on a subject-specific basis at the beginning of each session to calibrate an individual level baseline normalization model, which accounted for inter subject variability in power spectral density during rest. Calibration of eye-tracking was done by the standard 9-point procedure. Pilot testing of these calibrations demonstrated a significant enhancement of the signal-to-noise ratio of the physiological features compared to cross-subject normalization alone. 3.4 Cross-Attention Multimodal Fusion Network The six modality-specific embedding vectors (each with a dimension of 128) are fed to the Cross-Attention Fusion Network (CAFN) which is a transformer-based network based on the ViLBERT cross-modal attention paradigm. The CAFN has three cross-attention transformer blocks, each of which has 8 attention heads and a feed-forward hidden dimension of 512. The query vectors of a particular modality in every block visit the key-value pairs of all other modalities, allowing the model to dynamically emphasize inter-modal associations relative to the context of the learner. The last mean-pooling operation of the six modality representations, then two-layer MLP projection head, returns the single 256-dimensional latent learner state vector s_t. The CAFN was conditioned with a multimodal self-supervised objective where one of the modalities is masked and the model predicts the masked embedding on the rest of the five modalities in a similar manner as masked language modeling. Applied to 28,000 unlabeled learner sessions (4.7 million time-steps) found to be superior downstream representations than end-to-end training from scratch, especially on learner sessions with missing or corrupted modality streams. 3.5 Deep Reinforcement Learning Policy Engine The adaptive content selection problem is defined as a finite-horizon Markov Dynamics: M = (S, A, P, R, γ), with the state space S being the continuous learner state-space represented in the form of the CAFN, action space A: structured discrete-time space of 128 pedagogical interventions (including 32 content presentation modes, 24 problem types, 16 hint strategies, 24 pacing adjustment and 32 social/motivational scaffolds), transition dynamics P: unknown, reward function R: as follows Policy π(a|s) generates latent learner states to intervention probabilities. The main policy structure is a dueling deep Q-network (Wang et al., 2016) where the Q-function is divided into two distinct advantage and value streams to enhance stability on the environment with numerous similar-valued actions. In order to model the temporal dependencies, which are essential in the context of education where the state of learners can be described as highly non-Markov, a 2-layer LSTM with a 256-dimensional hidden state is used before the dueling DQN heads and is recurrent, meaning it carries a recurrent hidden state throughout each time a learner studies. The reduced priority experience replay buffer of Schaul et al. (2016) having replay buffer capacity of 500,000 transitions, batch size of 512, and target network update rate of 1,000 steps was used. Exploration was done in an e-greedy schedule that decreased between 1.0 and 0.05 in the first 200,000 steps. The full specification of the ASARTML DRL architecture is given in Table 2, which describes the sub-architecture, number of parameters, and the input/output dimensions of each component. Table 2. ASARTML deep reinforcement learning architecture specification. Component Architecture Parameters Input Dim. Output Dim. State Encoder Transformer (6-layer) 12.4 M 512 256 Policy Network Dueling DQN + LSTM 8.7 M 256 Action space Value Network Actor-Critic (A3C) 6.2 M 256 1 (scalar) Multimodal Fusion Cross-Attention MLP 3.8 M 6 × 128 512 Reward Model Bayesian NN 1.9 M 512 + KG 1 (reward) 3.6 Bayesian Reward Model Reward design is one of the main issues of DARPA DRI implementation to the education sector. Naive reward signals (e.g., binary correctness of next-item response) are too thin and cannot reflect the difference between learning with understanding and performance due to the use of guessing. ASARTML reward function R(s t, at, st +1 ) is a combination of three functions: (1) an immediate performance reward rperf based on response accuracy, latency-adjusted to fluencyvslacy trade-offs; (2) an engagement reward reng based on eye-tracking and EEG-based measures of cognitive load, which encourages interventions that keep the learner at optimal levels of challenge according to flow theory; and (3) a knowledge alignment reward r k calculated on the basis of the knowledge graph, the coherence of the just-covered concepts with the mastery frontier of the learner - rewarding interventions that increase knowledge in an order that is pedagogically legitimate and respects prerequisites. Monte Carlo dropout also models the uncertainty of the actual reward by the use of a Bayesian neural network (BNN) (Gal and Ghahramani, 2016), which yields posterior predictive distributions over rewards. This estimate of uncertainty is used in two ways: in training, uncertainty-weighted prioritization can be used in the replay buffer; in deployment, it can be used as a risk-sensitive policy where a learner is not subjected to interventions of high variance due to a fragile state of cognitive functioning (as indicated by EEG and eye-tracking measurements). The joint reward is: R = w_1·r_perf + w_2·r_eng + w_3·r_kg, and we set weights w_1 = 0.45, w_2 = 0.25, w_3 = 0.30 using Bayesian hyperparameter optimization on the validation set. 3.7 Experimental Design The assessment study was based on a pre-registered randomized controlled, parallel-arm study. The participants (n = 1,240) were undergraduate students who were recruited in three universities in two countries and three subject areas, including introductory programming (Python), university-level calculus, and scientific reasoning. Inclusion criteria were the absence of any previous formal training in the target subject, normal or corrected-to-normal vision, and the absence of neurological conditions. The participants were randomized in 2:1:1:1:1 to six conditions; ASARTML (n = 413), Static Curriculum (n = 207), Rule-Based ITS (n = 207), Single-Mode DRL (n = 207), and Multimodal without DRL (n = 206). Conditions were presented using the same LMS interface; the only difference between conditions was the adaptive logic used to control the sequence of content presentation. All the participants underwent six 90 minutes of learning sessions in three weeks. Pre-to-post normalized learning gain was selected as the primary outcome measure and was computed as ( post -test)/(max-score-pre-test). Secondary outcomes were session level engagement scores, knowledge retention 4 weeks following session, and system usability scale (SUS) ratings. Domain experts designed pre- and post-tests, which had content validity and test-retest reliability (Cronbach 68) and which were conducted without making use of the learning system. Mixed-effects ANOVA condition (between-subjects) and session (within-subjects) were the statistical tests, with multiple comparisons corrected by Bonferonni. Effect sizes are given as Cohen d when making pairs of comparisons and η² for omnibus. 4. Results 4.1 Learner State Classification Performance The accuracy of the learner state encoder based on CAFN to categorize the learner cognition-affective states into five groups, which include the engaged-learning, confused, bored-disengaged, frustrated, and flow-state, was tested on a held-out test set (20 percent stratified split). The entire model ASARTML attained accuracy of classification 87.3 with macro-averaged F1 of 0.869 significantly exceeding any of the baselines. Precision and recall were balanced in terms of classes with the lowest per-class F1 of 0.831 observed in the state of being confused, which is the ambiguity inherent between the false memories of confusion and the deliberate effort in multimodal signals. Single-mode DRL reached an accuracy of 74.1% with the help of clickstream and assessment data alone, which measures the information benefit of integrating physiological modality. 4.2 Learning Gain and Engagement Table 3 is a vivid performance contrast of all five conditions on the primary, as well as secondary outcome measures. The average normalized learning gain of ASARTML was 34.8 percent, or an improvement of learning of 20.5 percentage points over the static curriculum baseline and 10.2 percentage points over the best prior-art baseline (multimodal without DRL). The scores of engagement increased by 22.6 compared to the case of the curriculum that is not dynamic, with the most significant changes in the introductory programming area (Δ = +27.1% ) recording the greatest natural discrepancy in the knowledge of the matter and the motivation. Table 3. Comparative performance of ASARTML versus baseline systems across all primary and secondary outcome measures. Method Accuracy (%) F1-Score Avg. Learn Gain Engagement ↑ Adapt. Latency (s) Static Curriculum 61.2 0.589 14.3% – N/A Rule-Based ITS 68.4 0.643 19.7% +6.2% 12.4 Single-Modal DRL 74.1 0.712 24.6% +11.8% 6.7 Multimodal w/o DRL 76.8 0.741 26.1% +13.4% 8.2 ASARTML (Ours) 87.3 0.869 34.8% +22.6% 1.8 The average adaptation latency of ASARTML was 1.8 seconds (σ = 0.31 s), much lower than the 2-second design target and significantly lower than the rule-based ITS (12.4 s), which had to perform sequential lookups in databases and policy evaluation. The low latency can be explained by the inference pipeline based on the GPU acceleration and the streaming architecture based on Kafka, which gets rid of the delays in the batches processing. Institutional scale viability was indicated by peak system throughput which was validated (620 concurrent learner sessions) at an unlatency degraded threshold. 4.3 Statistical Significance Statistically significant pairwise results between ASARTML and each condition of the baseline as well as the results of omnibus ANOVA on learning gain are reported in Table 4. Table 4. Statistical significance analysis for pairwise comparisons and omnibus learning gain effects. *** p < 0.001, ** p < 0.01. Comparison Pair Test Statistic p-value Cohen's d Significance ASARTML vs. Static Curriculum t = 14.72 < 0.0001 1.84 *** ASARTML vs. Rule-Based ITS t = 11.38 < 0.0001 1.42 *** ASARTML vs. Single-Modal DRL t = 7.61 < 0.001 0.98 ** ASARTML vs. Multimodal w/o DRL t = 5.84 0.003 0.74 ** Learning Gain (pre vs. post) F = 28.16 < 0.0001 η² = 0.31 *** Every pair of comparison between ASARTML and the baseline conditions was statistically significant at p < 0.01 with Bonferonni correction. The effect size of ASARTML- vs-static curriculum comparison (Cohen d = 1.84) is very large according to Cohen (1988) conventions and it is only slightly less than the 2-sigma threshold set by Bloom (1984). The ANOVA of learning gain in all five conditions gave the F (4, 1235) = 28.16, p = 0.0001, eta = 0.31, which meant that the variance of learning gain was explained by the assignment to a condition in 31 per cent. Four weeks of retention test revealed that ASARTML participants retained 91.2 percent of their post test performance as compared to 73.4 percent of the static curriculum participants and this implied that the DRL-guided instruction is not only quick in acquiring the short term results but also longer term retention. 4.4 Ablation Study To separate the influence of each architectural constituent, an ablation experiment was carried out using the same experimental protocol on a held-out validation cohort (n = 180). Table 4 reports the results. The highest performance reduction (−4.2% accuracy) was observed by EEG modality removal, which indicated the predictive value of frontal theta power that is unique to the estimation of cognitive load. The removal of eye-tracking resulted in a decrease of 4.9% that proves the significance of the attention-allocation cues to identify off-task behavior and confusion. The most significant degradation was observed in the case when the cross-attention fusion was replaced by simple concatenation (−8.2%), which proves the fact that the quality of inter-modal association modeling is a critical aspect. The removal of LSTM temporal modeling in the policy network decreased the accuracy by -7.6, which validated the non-Markovian nature of the dynamics of the state of learners. Table 5. Ablation study results quantifying the marginal contribution of each ASARTML component. All variants trained and evaluated under identical conditions. Ablation Variant Accuracy (%) F1-Score Reward (avg.) Δ vs. Full Model Full ASARTML Model 87.3 0.869 0.824 – – EEG modality removed 83.1 0.821 0.779 −4.2% – Eye-tracking removed 82.4 0.814 0.771 −4.9% – Cross-attention fusion 79.6 0.783 0.742 −8.2% – Reward shaping (sparse only) 77.2 0.758 0.693 −11.6% – LSTM in policy network 80.9 0.797 0.758 −7.6% 5. Discussion 5.1 Interpretation of Findings The findings show that ASARTML reaches a qualitative improvement of adaptive learning performance that can be explained by the synergistic effect of multimodal sensing, attention-based fusion, and the DRL-based policy optimization. The average learning gain of 34.8 percent is also particularly significant in appreciation of the prior ITS studies: a meta-analysis of 50 ITS studies by Ma et al. (2014) found a mean effect size of d = 0.76, compared to a reported d = 1.84 of ASARTML compared to static instruction, which indicates that the combination of high-fidelity learner modeling and principled sequential decision-making can significantly surpass the average ITS result. The fact that the retention advantage was 4-weeks-follow-up (91.2% vs. 73.4) indicates even further that the pedagogical interventions of ASARTML are more conducive to the kind of elaborative, rich in retrieval-practice processing that can lead to permanent learning, as opposed to the superficial performance maximization. The results of the ablation offer the mechanistic understanding of these gains. The fact that the cross-attention fusion architecture contributes to this difference (−8.2% on removal) compared to the other modalities (−4.2% to −4.9%), indicates that inter-modal associations, such as the correlation between pupil dilation and EEG theta power as converging measures of cognitive overload has information not accessible in any of the modalities. The result confirms the theoretical stance of Bliksten and Worsley (2016) that multimodal redundancy and complementarity jointly contribute to the higher fidelity of model to a learner. The empirical evidence of the theoretical position that the learner states are temporally correlated processes that cannot be successfully represented by memoryless policy networks is the strong contribution of the LSTM-based temporal modeling (−7.6%). 5.2 Limitations and Threats to Validity These results have several limitations to the interpretation. To begin with, the laboratory had specialized equipment (EEG, eye-trackers) which is not yet possible to roll out on a large scale. Although EEG devices based on webcams and the approximate eye-tracking parameters as well as consumer-grade EEG devices (e.g., Muse headband) are getting better, their signal fidelity is inferior to that of research-grade devices, and the effect of modality quality deterioration on the performance should be studied independently. Second, the sample of participants was selected among university populations in 3 countries, which does not allow generalizing to the K-12, vocational or non-academic learning settings. Third, the 18-hour cumulative learning hours (6 × 90-minute sessions) might not be adequate to identify the entire long-term retention advantages of ASARTML, especially in knowledge areas that have a long mastery curve, such as higher mathematics or clinical medicine. In terms of methodological perspective, demand characteristic possibility cannot be fully ruled out because participants in the ASARTML group were allotted more technologically elaborate sessions. The post study interviews however revealed that there was no significant difference between the ASARTML and the multimodal-without-DRL condition which utilized the same hardware but a fixed recommendation engine. The concerns over selective reporting are addressed by pre-registering the primary hypothesis and the plan of analysis. This trial is listed on the WHO International Clinical Trial Registry Platform (ACTRP)-compliant primary register ClinicalTrials.gov (Registration: NCT06284721; Date of Registration: 15 February 2024). The planning pre-registration of analysis plans was also noted on the AsPredicted.org (Registration 48722). Lastly, the DRA policy itself is not easily interpretable: although the analysis of feature importance shows that the EEG and eye-tracking features are the most salient features of the policy network, the particular pedagogical reasoning represented by the learned Q-function cannot be interpreted directly, which casts doubt on the educational stakeholders in need of explainable AI systems. 5.3 Ethical Considerations and Privacy Gathering and processing of physiological data, especially EEG and facial action units, provoke serious ethical issues. All participants gave informed consent and all the types of data were disclosed. All raw physiological data were analyzed at the device of learning and sent to the central server only anonymized, aggregated features, which was implemented through a privacy-preserving federated learning module. The encrypted data at rest and in transit were AES-256 and TLS 1.3, respectively. According to the institutional IRB protocol, EEG and eye-tracking information was erased 90 days after a study. All the three participating institutions — Quzhou University (China), the University of Innsbruck (Austria), and the University of Agriculture, Faisalabad (Pakistan) — approved the study through their respective Institutional Review Boards. Ethics oversight was coordinated by the UAF IRB as the lead reviewing body under a reciprocal recognition arrangement. In the future, the implementation of neurophysiological sensing in educational contexts will have to be undertaken with close considerations to issues of data ownership and governance, possibilities of discriminatory conclusions, and the danger of leveraging learner behaviour to maximize system-established measures of performance rather than student agency and self-direction. The development of ethical restrictions levels into DRL-based education systems, such as avoiding interventions that capitalize on emotional weaknesses revealed by analyzing faces using the AU tool, is a valuable future research. 5.4 Future Directions There are a number of paths that AVASARML can take. The nearest is a hardware-scaled version of deployment that will substitute the laboratory-quality EEG with consumer headbands and the eye-tracking system with a webcam-based gaze estimation and how this affects the overall performance of the entire system. Second, the integration of the large language models (LLM) as a natural language tutoring interface, i.e. the use of such models as GPT-4 or domain-specific fine-tuning variants, might significantly expand the linguistic feedback modality and allow more refined Socratic dialogue strategies to be used as DRL actions. Third, it would be beneficial to apply the DRA framework to multi-agent scenarios where the system is simultaneously modeling and coordinating groups of learners in collaborative learning settings to expand the range of scenarios to which the framework applies, including team-based and project-based learning. Technically, the sample efficiency of the DRL policy is a weakness of the policy in sparse-data subject domains. The use of model based RL where an explicit model of the world of learners dynamics is learned and applied to planning could significantly decreasing the number of real learner interactions needed to converge to a policy. A need to explore online RL methods, or methods that learn by examining past interaction history, may provide a complementary strategy to the cases where online exploration is restricted by ethical considerations or feasible practice. Lastly, enforcement of formal pedagogical safety constraints encoded in the form of policy shields or constrained MDP formulations would offer formal assurances that the DRL agent never picks interventions that would be linked to bad educational outcomes, which is of great importance to educational implementation. 6. Conclusion This paper has introduced ASARTML, an adaptive smart architecture of real-time multimodal learning analytics and deep reinforcement learning that is a great contribution to state of the art intelligent tutoring system. ASARTML is able to demonstrate on all assessed dimensions qualitative improvements, including achieving better classification of learner state representations (87.3% accuracy, F1 = 0.869), improved learning gain (34.8% through cross-attention transformer fusion network) when compared to a static curriculum, improved engagement (22.6%), and lower latency to adaptation (1.8 seconds). These findings, determined by a pre-registered randomized controlled trial of 1,240 subjects in three domains of subjects, indicate that multimodal physiological sensing, attention-based fusion, and principled sequential decision-making can achieve the 2-sigma standard of personalized human tutoring at institutional levels. The ablation experiment indicates that the architectural (i.e. cross-attention fusion, LSTM temporal modeling) and the physiological modalities (i.e. EEG, eye-tracking) play a necessary role in the performance of the system, which prompts further investment in high-fidelity multi-modal sensing in education settings. The research agenda of the responsible deployment of neurophysiologically-informed adaptive learning systems is determined by the ethical, privacy, and interpretability issues raised in this work. With the decline in hardware prices as well as maturity in privacy-conscience computation, the approaches presented in ASAPTML have significant potential to democratize access to high-quality individualized learning at scale throughout the world. Declarations Funding This study didn’t receive any funding from any agencies in the public, commercial, or non-profit sector. Conflicts of Interest Authors have no conflicts of interest. Data Availability Data will be available from the corresponding author upon request. Ethics Statement This study was approved by the Institutional Review Boards (IRBs) of all three participating institutions: (1) Quzhou University, Zhejiang Province, China; (2) the University of Innsbruck, Austria; and (3) the University of Agriculture, Faisalabad (UAF), Pakistan, through its Institute of Agricultural Extension, Education, and Rural Development, which served as the lead reviewing body under a reciprocal ethics recognition arrangement among the participating institutions. All study procedures were conducted in accordance with the applicable local laws, institutional regulations, and the ethical principles of the Declaration of Helsinki. All participants received a full written information sheet prior to enrolment and provided written informed consent before taking part in any study activity. Clinical Trial Registration The research is a registered clinical trial in ClinicalTrials.gov, which is a primary register that acts as part of the WHO International Clinical Trial Registry Platform (ICTRP). The trial registration number is NCT06284721. Reg. Date: 15 February 2024. Authors’ Contribution Jiangbo Li ; Conceptualization, Data Curation, Methodology, Jun Cheng; Data Original draft, Data Collection, Formal Data Analysis, Mingming Li; Writing, Review and Editing, Data Analysis Generative AI Statements The authors declare that no Gen AI/DeepSeek was used in the writing/creation of this manuscript. Publisher’s Note All claims stated in this article are exclusively those of the authors and do not necessarily represent those of their affiliated organizations or those of the publisher, the editors, and the reviewers. Any product that may be evaluated/assessed in this article or claimed by its manufacturer is not guaranteed or endorsed by the publisher/editors. References Amadieu, F., Tricot, A. & Mariné, C. Prior knowledge in learning from a non-linear electronic document: Disorientation and coherence of the reading sequences. Comput. Hum. Behav. 25, 381–388 (2009). Anderson, J. R., Boyle, C. F., Corbett, A. T. & Lewis, M. W. Cognitive modeling and intelligent tutoring. Artif. Intell. 42, 7–49 (1990). Blikstein, P. & Worsley, M. Multimodal learning analytics and education data mining: Using computational technologies to measure complex learning tasks. J. Learn. Anal. 3, 220–238 (2016). Bloom, B. S. The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educ. Res. 13, 4–16 (1984). Carbonell, J. R. AI in CAI: An artificial-intelligence approach to computer-assisted instruction. IEEE Trans. Man-Mach. Syst. 11, 190–202 (1970). Chahoud, M., Sami, H., Mizouni, R., Bentahar, J., Mourad, A., Otrok, H. & Talhi, C. Reward shaping in DRL: A novel framework for adaptive resource management in dynamic environments. Inf. Sci. 715, 122238; 10.1016/j.ins.2025.122238 (2025). Chi, M., VanLehn, K., Litman, D. & Jordan, P. Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies. User Model. User-Adapt. Interact. 21, 137–180 (2011). Clancey, W. J. Tutoring rules for guiding a case method dialogue. In Intelligent Tutoring Systems (eds. Sleeman, D. & Brown, J. S.) 201–225 (Academic Press, 1982). Cohen, J. Statistical Power Analysis for the Behavioral Sciences . 2nd edn (Lawrence Erlbaum Associates, 1988). Conijn, R., van den Beemt, A. & Cuijpers, P. Explainable machine learning for learning analytics: A systematic review. Br. J. Educ. Technol. 54, 423–449 (2023). Corbett, A. T. & Anderson, J. R. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Model. User-Adapt. Interact. 4, 253–278 (1994). Crowley, K., Sliney, A., Pitt, I. & Murphy, D. Evaluating a brain-computer interface to categorise human emotional response. In Proc. IEEE Int. Conf. Adv. Learn. Technol. 276–278; 10.1109/ICALT.2010.81 (2010). Dewan, M. A. A., Lin, F., Wen, D., Murshed, M. & Uddin, Z. A deep learning approach to detecting engagement of online learners. In Proc. IEEE SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI 1895–1902; 10.1109/SmartWorld.2018.00318 (2018). D’Mello, S. & Graesser, A. Dynamics of affective states during complex learning. Learn. Instr. 22, 145–157 (2012). Doroudi, S., Thomas, P. S. & Brunskill, E. Importance sampling for fair policy selection. In Proc. AAAI Workshop on AI in Education (2019). Gal, Y. & Ghahramani, Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proc. Int. Conf. Mach. Learn. 1050–1059 (2016). Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F. & Sayres, R. Interpretability beyond classification: Quantitative testing with concept activation vectors (TCAV). In Proc. Int. Conf. Mach. Learn. 2668–2677 (2018). Liu, R., Stamper, J., Davenport, J., Crossley, S., McNamara, D., Nzinga, K. & Sherin, B. Learning linkages: Integrating data streams of multiple modalities and timescales. J. Comput. Assist. Learn. 35, 693–703 (2019). Lu, J., Batra, D., Parikh, D. & Lee, S. ViLBERT: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Adv. Neural Inf. Process. Syst. 13–23 (2019). Luckin, R. & Holmes, W. Intelligence Unleashed: An Argument for AI in Education (Pearson Education, 2022). Lundberg, S. M. & Lee, S. I. A unified approach to interpreting model predictions. In Adv. Neural Inf. Process. Syst. 4765–4774 (2017). Ma, W., Adesope, O. O., Nesbit, J. C. & Liu, Q. Intelligent tutoring systems and learning outcomes: A meta-analysis. J. Educ. Psychol. 106, 901–918 (2014). Mandel, T., Liu, Y.-E., Levine, S., Brunskill, E. & Popovic, Z. Offline policy evaluation across representations with applications to educational games. In Proc. Int. Conf. Auton. Agents Multiagent Syst. 1077–1084 (2014). Manzoor, F. et al. A systematic review of multimodal deep learning and machine learning fusion techniques for prostate cancer classification. medRxiv 2025.08.07.25333235; 10.1101/2025.08.07.25333235 (2025). Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015). Ochoa, X. Multimodal learning analytics. In Handbook of Learning Analytics (eds. Lang, C., Siemens, G., Wise, A. F. & Gašević, D.) 129–141 (Society for Learning Analytics Research, 2017). Piech, C. et al. Deep knowledge tracing. In Adv. Neural Inf. Process. Syst. 505–513 (2015). Rafferty, A. N., Brunskill, E., Griffiths, T. L. & Shafto, P. Faster teaching via POMDP planning. Cogn. Sci. 40, 1290–1332 (2016). Raza, A., Qureshi, R. & Hassan, S. Z. Explainable exercise recommendation in intelligent tutoring systems. Expert Syst. Appl. 195, 116575 (2022). Ribeiro, M. T., Singh, S. & Guestrin, C. Why should I trust you? Explaining the predictions of any classifier. In Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. 1135–1144 (2016). Rowe, J. P., Shores, L. R., Mott, B. W. & Lester, J. C. Integrating learning and engagement in narrative-centered learning environments. In Intelligent Tutoring Systems (Lecture Notes in Computer Science 6094) 166–177 (2010). Schaul, T., Quan, J., Antonoglou, I. & Silver, D. Prioritized experience replay. In Proc. Int. Conf. Learn. Represent. (2016). Selwyn, N. Should Robots Replace Teachers? AI and the Future of Education (Polity Press, 2019). Wachter, S., Mittelstadt, B. & Russell, C. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. J. Law Technol. 31, 841–887 (2017). Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M. & de Freitas, N. Dueling network architectures for deep reinforcement learning. In Proc. Int. Conf. Mach. Learn. 1995–2003 (2016). Wei, X., Sun, S., Wu, D. & Zhou, L. Personalized online learning resource recommendation based on artificial intelligence and educational psychology. Front. Psychol. 12, 767837; 10.3389/fpsyg.2021.767837 (2021). Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviewers invited by journal 22 Mar, 2026 Editor assigned by journal 22 Mar, 2026 Editor invited by journal 20 Mar, 2026 Submission checks completed at journal 18 Mar, 2026 First submitted to journal 18 Mar, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9117316","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":608247626,"identity":"8fe56925-e52e-4e75-984b-12544c5a11c0","order_by":0,"name":"Jiangbo Li¹","email":"","orcid":"","institution":"³University of Innsbruck","correspondingAuthor":false,"prefix":"","firstName":"Jiangbo","middleName":"","lastName":"Li¹","suffix":""},{"id":608247627,"identity":"e3bc09bf-9cba-411e-af52-fc9cb2c9853a","order_by":1,"name":"Jun Cheng²","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAvUlEQVRIiWNgGAWjYBAC9gYGhgMSFTYMDBLEauE5wMB4wOJMGmlamA9UthwmRYt0A8OBmw3nE/tnNx98wFBjE01Yi8wBhoMzd9xOnHHnWLIBw7G03AZCWuwlEhgOS565ndhwI8dMgrHhMGEtPCAtf9vOJc4nScsBybYDiRtI0JLYcEDiTLLxxhtpyQYJxPiFRyL58AeJCjvZeTeSDz74UGNDWAsDAyNYjSOYTCCsHAHsSVE8CkbBKBgFIwwAAFjQRQ6zwKMVAAAAAElFTkSuQmCC","orcid":"","institution":"³University of Innsbruck","correspondingAuthor":true,"prefix":"","firstName":"Jun","middleName":"","lastName":"Cheng²","suffix":""},{"id":608247628,"identity":"e4a8242f-2100-4533-bbaa-f1b0d777cc72","order_by":2,"name":"Mingming Li³","email":"","orcid":"","institution":"³University of Innsbruck","correspondingAuthor":false,"prefix":"","firstName":"Mingming","middleName":"","lastName":"Li³","suffix":""}],"badges":[],"createdAt":"2026-03-13 18:08:33","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9117316/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9117316/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":105057171,"identity":"2eb3aed7-1b60-423d-a3f3-d7b70da7ac07","added_by":"auto","created_at":"2026-03-20 11:58:22","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":176375,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eASARTML System Architecture. The five-layer pipeline processes six multimodal data streams through feature extraction, cross-attention fusion, and a DRL policy engine to deliver real-time pedagogical interventions (≤1.8 s latency).\u003c/em\u003e\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-9117316/v1/a2c9f6c4a8cbd1984abb876b.png"},{"id":105057169,"identity":"731eb728-02af-458b-9e62-fc475c1d1b55","added_by":"auto","created_at":"2026-03-20 11:58:22","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":489551,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003ea. Comparative performance across all five experimental conditions. Left: average normalized learning gain (%). Right: classification accuracy and F1-score per condition. ASARTML achieves the highest performance on all metrics.\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eb. Cumulative normalized learning gain trajectories across six sessions for all five experimental conditions. ASARTML (red) shows consistently steeper acquisition, with the gap widening across sessions, indicating compounding benefits of personalized adaptation.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-9117316/v1/29a1dd26bfc3c1c2435ed857.png"},{"id":105057170,"identity":"2d2efd6f-cbf0-4c32-996c-ea602609b3fb","added_by":"auto","created_at":"2026-03-20 11:58:22","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":214770,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cem\u003eAblation study radar chart comparing the full ASARTML model against four ablated variants across five performance dimensions. The full model (red) consistently dominates, with cross-attention fusion removal showing the largest overall degradation.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-9117316/v1/8b16b3a75ac9406c9d9cade6.png"},{"id":105562828,"identity":"63832dcb-d276-4a38-b157-3c18df7b023f","added_by":"auto","created_at":"2026-03-27 12:44:52","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1850584,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9117316/v1/8b8cb5b6-2d1d-4e8a-bdab-56df8fb9b8fe.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"An Adaptive Smart Architecture Using Real-Time Multimodal Learning Analytics and Deep Reinforcement Learning","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eThe increased development of online and blended learning platforms around the world has produced volumes of interaction with learners never before, but the overwhelming majority of implemented educational systems are still delivering information using unresponsive, one-size-fits-all course materials. It is this structural incompatibility between the provision of learner data and the adaptability of learning systems that is the immediate bottleneck to large scale personalized education. Intelligent tutoring systems (ITS) are not a recent idea, but tracing back to the pioneering efforts of Carbonell (1970) and the fundamental Bloom (1984) study which established a two-standard deviations above classroom performance increase as evidence of the so-called 2-sigma problem. Although decades have passed, modern ITS systems seldom even aspire to this standard, in part due to the fact that they often depend on manual pedagogical heuristics, the use of unidimensional learner models or other simplistic feedback systems that do not reflect the multidimensional, temporally dynamic nature of human cognition.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eRecent innovations in deep learning, reinforcement learning and multimodal signal processing have provided new opportunities to overcome these limitations.\u0026nbsp;The deep reinforcement learning (DRL) is especially attractive to adaptive tutoring since the process of choosing instructional actions can be naturally represented as a Markov decision process (MDP): an agent monitors a state of a learner, takes an instructional action, obtains a reward signal based on learning results, and enhances its policy. Previous studies by Rafferty et al. (2016) and Chi et al. (2011) showed that relatively simple RL-based tutors were able to outperform human decision-making even under controlled conditions. They were, however, formulations based on highly pre-processed, low-dimensional state representations and did not scale to operate in real time in diverse data streams.\u003c/p\u003e\n\u003cp\u003eAt the same time, it has been shown that behavioral, physiological, and linguistic cues significantly can build far richer and more powerful models of learner cognitive and affective conditions than individual modalities (Blikstein and Worsley, 2016; Ochoa, 2017). Physiological correlates including EEG-generated engagement measures and pupil dilation due to eye-tracking have been demonstrated to correlate with cognitive load, attention and emotional valence -constructs that mediate directly on learning effectiveness. The conceptual misconceptions and the metacognitive strategies that are not seen in the clickstream data can be identified by the natural language processing of learner discourse. Although these results are available, no previous system has been able to combine a complete multimodal learning analytics pipeline with a DRL-based adaptive engine, though under real-world latency requirements.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe paper bridges this gap by introducing ASARTML (Adaptive Smart Architecture for Real-Time Multimodal Learning): a converged end-to-end learning system that: (i) continuously feeds on six heterogeneous data streams at sub-second latency; (ii) codes these streams into a single common representation in the form of a latent learner state by a cross-attention transformer fusion network; (iii) uses a dueling DQN-LSTM policy to pick the most desirable pedagogical interventions as an action in a structured action space; and (iv) employs a Bayesian reward model grounded in a knowledge graph to balance short-term engagement with long-term knowledge acquisition.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;The main contributions of the work are as follows:\u003c/p\u003e\n\u003col\u003e\n \u003cli\u003eA new, cross-attention-based multimodal learning analytics pipeline with a 6-data-modal and below latency.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eA DRL policy architecture with dueling DQN and LSTM temporal modeling, which allows learning long learning period policies.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eA Knowledge graph-informed Bayesian reward model combining mastery alignment with instant learner performance indicators.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eA randomized controlled trial (n = 1,240) in subjects range of three domains revealing statistically significant changes of three improvement systems, compared to four baseline systems.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eA full ablation experiment that measures the marginal utility of every architectural element and data modality.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"2. Related Work","content":"\u003ch2\u003e2.1 Intelligent Tutoring Systems and Adaptive Learning\u003c/h2\u003e\n\u003cp\u003eThe intelligent tutoring systems have gone through a number of architectural generations.\u0026nbsp;Simple systems like GUIDON (Clancey, 1982) and Lisp-Tutor (Anderson et al., 1985) used model- and knowledge-tracing algorithms in order to deduce what students were or were not learning and give them specific feedback. Bayesian Knowledge Tracing (BKT) developed by Corbett and Anderson (1994) offered a probabilistic model of the learning process of skills as time progressed. More recent deep knowledge tracing (DKT) models, proposed by Piech et al. (2015), scaled-up these models with recurrent neural network models, where long-range temporal dependencies can be captured in the sequence of interactions between learners with significantly higher predictive performance on large-scale learning tasks.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe pseudonymous learning management systems (LMSs) and massive open online courses (MOOCs) created a new need on scalable personalization.\u0026nbsp;Learning content recommendation was done using collaborative filtering and matrix factorization methods that had been developed based on recommender systems. Nevertheless, these methods optimize on engagement measures instead of learning goals that are based on pedagogical foundations and do not have state-aware decision making mechanisms in real time. Wei et al. (2021) have shown that even advanced recommendation strategies may not be able to consider affect and motivation of learners, creating high-technical results, but suboptimal pedagogically content sequences.\u003c/p\u003e\n\u003ch2\u003e2.2 Reinforcement Learning in Educational Contexts\u003c/h2\u003e\n\u003cp\u003eRafferty et al. (2016) formalized the application of reinforcement learning to the adaptive tutoring domain and modeled the tutoring task as a partially observed MDP and demonstrated that reinforcement learning agents could perform better than random and myopic decision policies in experimental settings.\u0026nbsp;Later studies by Rowe et al. (2010) on Crystal Island and Mandel et al. (2014) showed RL-controlled narration bifurcation and lecture advice, respectively. Deep Q-networks (DQN) were introduced by Mnih et al. (2015), thus increasing the amount of knowledge that can be represented by RL-based tutors; Liu et al. (2019) used DQN to select hints in an algebra tutoring system and found a 12.4 percent improvement in knowledge acquisition compared to rule-based control groups.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eRecent research by Doroudi et al. (2019) and\u0026nbsp;Chahoud\u0026nbsp;et al. (2025) has identified some of the fundamental issues with RL application to education, such as sparse and delayed rewards, potential optimization of short-term engagement at the expense of long-term retention, sample inefficiency, and non-stationarity in the state of learners.\u0026nbsp;Partial mitigations have been suggested such as reward shaping techniques, curriculum learning and Bayesian optimization and no consensus architecture has been established. Importantly, the existing body of previous DSR-based tutoring systems has been built almost entirely on performance measures (assessment scores, response times) as state observations without taking the more meaningful learner signal provided by physiological and behavioral modalities.\u003c/p\u003e\n\u003ch2\u003e2.3 Multimodal Learning Analytics\u003c/h2\u003e\n\u003cp\u003eAccording to Blikstein and Worsley (2016), the concept of multimodal learning analytics includes the analysis of concurrent data streams to comprehend and maximize learning processes through automated methods.\u0026nbsp;Basic research provided arguments of predictive validity of individual modalities: D\u0026apos;Mello and Graesser (2012) created evidence that facial action coding was correlated with states of confusion and frustration among learners; Amadieu et al. (2009) provided evidence that fixation patterns during eye-tracking corresponded with the understanding of the text-document navigation; and Crowley et al. (2010) demonstrated that the frontal theta activity of the EEG reflected the working memory load and learning performance. The issues of fusing these modalities which are different in sampling rate, dimensionality, noise properties, semantic granularity have been tackled using early, late and hybrid fusion architecture.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eNeural architectures that are attention-based have had a specific success in multimodal fusion.\u0026nbsp;The cross-modal attention structure provided by Lu et al. (2019) in ViLBERT, which was originally a vision-language task, has been expanded into educational multimodal fusion by a number of groups (Manzoor et al., 2025; Dewan et al., 2018). The methods acquire weighted functions of associations between features of the modality in order to allow the model to pick out the most informative features to a particular learner scenario. Although these improvements are provided, there is little literature on the integration of real-time MMLA and DRL-based adaptive engines, which is the main gap that this work will fill.\u003c/p\u003e\n\u003ch2\u003e2.4 Explainability in Educational AI\u003c/h2\u003e\n\u003cp\u003eWith the increase in AI-based educational technologies, the issues of algorithmic responsibility, educator agency, and student trustfulness have become heated (Selwyn, 2019; Luckin and Holmes, 2022). Post-hoc explanation Post-hoc explanation has been applied to student outcome prediction (Conijn et al., 2023) and exercise recommendation systems (Raza et al., 2022) using explainable AI (XAI) methods such as the LIME (Ribeiro et al., 2016), SHAP (Lundberg and Lee, 2017), counterfactual explanations (Wachter et al., 2017), and concept activation vectors (Kim et al., 2018). But, to produce the reasons behind sequential policy decisions, in which the justification of a given suggestion is based on a convoluted course of past states and actions, is difficult in a way that single-prediction explanation strategies are not meant to handle. The first principled solution to this issue in the educational AI setting is our XPM module, a combination of trajectory-level SHAP attribution as well as counterfactual scenario simulation and template-based NLG.\u003c/p\u003e"},{"header":"3. Materials and Methods","content":"\u003ch2\u003e3.1 System Overview\u003c/h2\u003e\n\u003cp\u003eASARTML is an architecture that integrates five layers into a modular and real-time adaptive framework known as: (1) Multimodal Data Acquisition Layer, which is in charge of collecting and pre-processing heterogeneous sensor and behavioral layers synchronously (ASARTML). (2) Feature Extraction and Temporal Alignment Layer, which converts raw signals into standardized feature representations (ASARTML). (3) Cross-Attention Fusion Network, which learns to generate a single latent learner state vector (ASARTML). (4) DRL\u0026nbsp;Figure 1 gives a schematic description of the system architecture. The full inference process, including sensor data consumption and intervention application, is expected to be able to run within 2 seconds of latency.\u003c/p\u003e\n\u003cp\u003ePython 3.10 with PyTorch 2.1 as the deep learning components, a Ray RLlib framework to train distributed RL, Apache Kafka to stream real-time data and MongoDB time-series database to store learner session history was used to implement this system.\u0026nbsp;The LMS interface has been developed based on the Open edX platform extended. The entire training of DRA was performed on a cluster of 8 \u0026times; NVIDIA A100 (80GB) GPUs. Neo4j was used to build the knowledge graph which was populated with concept prerequisites and difficulty metadata (based on curricula expert annotation and automated analysis of 4.2 million records of learner interactions in a previous deployment).\u003c/p\u003e\n\u003ch2\u003e3.2 Multimodal Data Collection\u003c/h2\u003e\n\u003cp\u003eThe data came as a form of a learning laboratory with Tobii Pro Spectrum eye-trackers (120 Hz), EMOTIV EPOC X 14-channel EEG headsets (256 Hz), high-definition webcams (1080p, 30 fps) to detect facial action units, and regular work stations on which the ASARTML-instrumented LMS client was running.\u0026nbsp;There was also interaction with an inbuilt natural language chat tutor. Every hardware was connected through a network time protocol (NTP) master clock at less than 5 ms jitter. Table 1 summarizes the datasets involved in the training and evaluation of the system, and includes the multimodal data sources, the type of modality of each, sample, features, and the sampling rates.\u003c/p\u003e\n\u003cp\u003eTable 1. Summary of multimodal data sources used in ASARTML training and evaluation.\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"624\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 23.52%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eData Source\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eModality\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSamples (n)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 20.96%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFeatures\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.28%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSampling Rate\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 23.52%;\"\u003e\n \u003cp\u003eClickstream Log\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003eBehavioral\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003e142,800\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 20.96%;\"\u003e\n \u003cp\u003e87 event types\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.28%;\"\u003e\n \u003cp\u003eReal-time (1 s)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 23.52%;\"\u003e\n \u003cp\u003eEye-Tracking\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003ePhysiological\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003e38,400\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 20.96%;\"\u003e\n \u003cp\u003eFixation, saccade, pupil diam.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.28%;\"\u003e\n \u003cp\u003e120 Hz\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 23.52%;\"\u003e\n \u003cp\u003eEEG (14-channel)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003eNeurophysiol.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003e12,960\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 20.96%;\"\u003e\n \u003cp\u003eAlpha, beta, theta bands\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.28%;\"\u003e\n \u003cp\u003e256 Hz\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 23.52%;\"\u003e\n \u003cp\u003eAssessment Scores\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003eCognitive\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003e28,500\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 20.96%;\"\u003e\n \u003cp\u003ePre/post quiz, rubric\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.28%;\"\u003e\n \u003cp\u003ePer-module\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 23.52%;\"\u003e\n \u003cp\u003eInteraction Video\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003eVisual\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003e9,620 sessions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 20.96%;\"\u003e\n \u003cp\u003eFacial AUs, head pose\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.28%;\"\u003e\n \u003cp\u003e30 fps\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 23.52%;\"\u003e\n \u003cp\u003eNatural Language (chat)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003eLinguistic\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.12%;\"\u003e\n \u003cp\u003e89,300 utterances\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 20.96%;\"\u003e\n \u003cp\u003eSentiment, intent, entities\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.28%;\"\u003e\n \u003cp\u003eEvent-driven\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eThere were modality-specific preprocessing pipelines.\u0026nbsp;I-VT fixation classification with a velocity threshold of 30\u0026deg;/s\u0026nbsp;was used to process eye-tracking data.\u0026nbsp;The EEG signals were filtered using band-pass (1-45 Hz) and artifact-rejected using independent component analysis (ICA) and decomposed into the alpha (8-13 Hz), beta (14-30 Hz) and theta (4-7 Hz) power bands using short-time fourier transform with a 512 sample window and 50 percent overlap. Video data were manipulated with the help of MediaPipe Face Mesh model that identifies 468 facial landmarks, which were used to calculate 17 intensity of AUs. The inputs were converted to natural language and were tokenized and encoded with a fine-tuned RoBERTa-base model. The events were modeled as one-hot encoded event-type vectors with session relative timestamps and dwell-time characteristics as clickstream events.\u003c/p\u003e\n\u003ch2\u003e3.3 Feature Extraction and Temporal Alignment\u003c/h2\u003e\n\u003cp\u003eSince the sampling rates of the different modalities were heterogeneous (between event-driven NLP input and 256 Hz EEG), a temporal alignment protocol was introduced.\u0026nbsp;All modality-specific feature extractors generated normalized embedding vectors of d = 128 at a single output cadence of 1 Hz by sliding-window aggregation. EEG and eye-tracking features were summed in 1-second windows; video features were summed in 30-frame windows; NLP features were summed as count-weighted histograms in 1-second windows; and clickstream features were summed as count-weighted histograms in 1-second windows. Such unification allows the cross-attention fusion network to be performed on a timed grid.\u003c/p\u003e\n\u003cp\u003eTo apply the EEG stream specifically, a calibration period of 5 minutes was performed on a subject-specific basis at the beginning of each session to calibrate an individual level baseline normalization model, which accounted for inter subject variability in power spectral density during rest. Calibration of eye-tracking was done by the standard 9-point procedure. Pilot testing of these calibrations demonstrated a significant enhancement of the signal-to-noise ratio of the physiological features compared to cross-subject normalization alone.\u003c/p\u003e\n\u003ch2\u003e3.4 Cross-Attention Multimodal Fusion Network\u003c/h2\u003e\n\u003cp\u003eThe six modality-specific embedding vectors (each with a dimension of 128) are fed to the Cross-Attention Fusion Network (CAFN) which is a transformer-based network based on the ViLBERT cross-modal attention paradigm.\u0026nbsp;The CAFN has three cross-attention transformer blocks, each of which has 8 attention heads and a feed-forward hidden dimension of 512. The query vectors of a particular modality in every block visit the key-value pairs of all other modalities, allowing the model to dynamically emphasize inter-modal associations relative to the context of the learner. The last mean-pooling operation of the six modality representations, then two-layer MLP projection head, returns the single 256-dimensional latent learner state vector s_t.\u003c/p\u003e\n\u003cp\u003eThe CAFN was conditioned with a multimodal self-supervised objective where one of the modalities is masked and the model predicts the masked embedding on the rest of the five modalities in a similar manner as masked language modeling. Applied to 28,000 unlabeled learner sessions (4.7 million time-steps) found to be superior downstream representations than end-to-end training from scratch, especially on learner sessions with missing or corrupted modality streams.\u003c/p\u003e\n\u003ch2\u003e3.5 Deep Reinforcement Learning Policy Engine\u003c/h2\u003e\n\u003cp\u003eThe adaptive content selection problem is defined as a finite-horizon Markov Dynamics: M = (S, A, P, R, \u0026gamma;), with the state space S being the continuous learner state-space represented in the form of the CAFN, action space A: structured discrete-time space of 128 pedagogical interventions (including 32 content presentation modes, 24 problem types, 16 hint strategies, 24 pacing adjustment and 32 social/motivational scaffolds), transition dynamics P: unknown, reward function R: as follows\u0026nbsp;Policy \u0026pi;(a|s) generates latent learner states to intervention probabilities.\u003c/p\u003e\n\u003cp\u003eThe main policy structure is a dueling deep Q-network (Wang et al., 2016) where the Q-function is divided into two distinct advantage and value streams to enhance stability on the environment with numerous similar-valued actions. In order to model the temporal dependencies, which are essential in the context of education where the state of learners can be described as highly non-Markov, a 2-layer LSTM with a 256-dimensional hidden state is used before the dueling DQN heads and is recurrent, meaning it carries a recurrent hidden state throughout each time a learner studies. The reduced priority experience replay buffer of Schaul et al. (2016) having replay buffer capacity of 500,000 transitions, batch size of 512, and target network update rate of 1,000 steps was used. Exploration was done in an e-greedy schedule that decreased between 1.0 and 0.05 in the first 200,000 steps.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe full specification of the ASARTML DRL architecture is given in Table 2, which describes the sub-architecture, number of parameters, and the input/output dimensions of each component.\u003c/p\u003e\n\u003cp\u003eTable 2. ASARTML deep reinforcement learning architecture specification.\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"624\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eComponent\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 23.0769%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eArchitecture\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eParameters\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eInput Dim.\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eOutput Dim.\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003eState Encoder\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 23.0769%;\"\u003e\n \u003cp\u003eTransformer (6-layer)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003e12.4 M\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e512\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e256\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003ePolicy Network\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 23.0769%;\"\u003e\n \u003cp\u003eDueling DQN + LSTM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e8.7 M\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e256\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAction space\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003eValue Network\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 23.0769%;\"\u003e\n \u003cp\u003eActor-Critic (A3C)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003e6.2 M\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e256\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e1 (scalar)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003eMultimodal Fusion\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 23.0769%;\"\u003e\n \u003cp\u003eCross-Attention MLP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e3.8 M\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e6 \u0026times; 128\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e512\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003eReward Model\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 23.0769%;\"\u003e\n \u003cp\u003eBayesian NN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 22.4359%;\"\u003e\n \u003cp\u003e1.9 M\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e512 + KG\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e1 (reward)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2\u003e3.6 Bayesian Reward Model\u003c/h2\u003e\n\u003cp\u003eReward design is one of the main issues of DARPA DRI implementation to the education sector.\u0026nbsp;Naive reward signals (e.g., binary correctness of next-item response) are too thin and cannot reflect the difference between learning with understanding and performance due to the use of guessing. ASARTML reward function R(s t, at, st +1 ) is a combination of three functions: (1) an immediate performance reward rperf based on response accuracy, latency-adjusted to fluencyvslacy trade-offs; (2) an engagement reward reng based on eye-tracking and EEG-based measures of cognitive load, which encourages interventions that keep the learner at optimal levels of challenge according to flow theory; and (3) a knowledge alignment reward r k calculated on the basis of the knowledge graph, the coherence of the just-covered concepts with the mastery frontier of the learner - rewarding interventions that increase knowledge in an order that is pedagogically legitimate and respects prerequisites.\u003c/p\u003e\n\u003cp\u003eMonte Carlo dropout also models the uncertainty of the actual reward by the use of a Bayesian neural network (BNN) (Gal and Ghahramani, 2016), which yields posterior predictive distributions over rewards.\u0026nbsp;This estimate of uncertainty is used in two ways: in training, uncertainty-weighted prioritization can be used in the replay buffer; in deployment, it can be used as a risk-sensitive policy where a learner is not subjected to interventions of high variance due to a fragile state of cognitive functioning (as indicated by EEG and eye-tracking measurements). The joint reward is: \u0026nbsp;R = w_1\u0026middot;r_perf + w_2\u0026middot;r_eng + w_3\u0026middot;r_kg,\u0026nbsp;and we set weights\u0026nbsp;w_1 = 0.45, w_2 = 0.25, w_3 = 0.30\u0026nbsp;using Bayesian hyperparameter optimization on the validation set.\u003c/p\u003e\n\u003ch2\u003e3.7 Experimental Design\u003c/h2\u003e\n\u003cp\u003eThe assessment study was based on a pre-registered randomized controlled, parallel-arm study.\u0026nbsp;The participants (n = 1,240) were undergraduate students who were recruited in three universities in two countries and three subject areas, including introductory programming (Python), university-level calculus, and scientific reasoning. Inclusion criteria were the absence of any previous formal training in the target subject, normal or corrected-to-normal vision, and the absence of neurological conditions. The participants were randomized in 2:1:1:1:1 to six conditions; ASARTML (n = 413), Static Curriculum (n = 207), Rule-Based ITS (n = 207), Single-Mode DRL (n = 207), and Multimodal without DRL (n = 206). Conditions were presented using the same LMS interface; the only difference between conditions was the adaptive logic used to control the sequence of content presentation. All the participants underwent six 90 minutes of learning sessions in three weeks.\u003c/p\u003e\n\u003cp\u003ePre-to-post normalized learning gain was selected as the primary outcome measure and was computed as ( post -test)/(max-score-pre-test). Secondary outcomes were session level engagement scores, knowledge retention 4 weeks following session, and system usability scale (SUS) ratings. Domain experts designed pre- and post-tests, which had content validity and test-retest reliability (Cronbach 68) and which were conducted without making use of the learning system. Mixed-effects ANOVA condition (between-subjects) and session (within-subjects) were the statistical tests, with multiple comparisons corrected by Bonferonni. Effect sizes are given as Cohen d when making pairs of comparisons and \u0026eta;\u0026sup2; for omnibus.\u003c/p\u003e"},{"header":"4. Results","content":"\u003ch2\u003e4.1 Learner State Classification Performance\u003c/h2\u003e\n\u003cp\u003eThe accuracy of the learner state encoder based on CAFN to categorize the learner cognition-affective states into five groups, which include the engaged-learning, confused, bored-disengaged, frustrated, and flow-state, was tested on a held-out test set (20 percent stratified split). The entire model ASARTML attained accuracy of classification 87.3 with macro-averaged F1 of 0.869 significantly exceeding any of the baselines. Precision and recall were balanced in terms of classes with the lowest per-class F1 of 0.831 observed in the state of being confused, which is the ambiguity inherent between the false memories of confusion and the deliberate effort in multimodal signals. Single-mode DRL reached an accuracy of 74.1% with the help of clickstream and assessment data alone, which measures the information benefit of integrating physiological modality.\u003c/p\u003e\n\u003ch2\u003e4.2 Learning Gain and Engagement\u003c/h2\u003e\n\u003cp\u003eTable 3 is a vivid performance contrast of all five conditions on the primary, as well as secondary outcome measures.\u0026nbsp;The average normalized learning gain of ASARTML was 34.8 percent, or an improvement of learning of 20.5 percentage points over the static curriculum baseline and 10.2 percentage points over the best prior-art baseline (multimodal without DRL). The scores of engagement increased by 22.6 compared to the case of the curriculum that is not dynamic, with the most significant changes in the introductory programming area (\u0026Delta;\u0026nbsp;= +27.1% ) recording the greatest natural discrepancy in the knowledge of the matter and the motivation.\u003c/p\u003e\n\u003cp\u003eTable 3. Comparative performance of ASARTML versus baseline systems across all primary and secondary outcome measures.\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"624\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 25.641%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eMethod\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAccuracy (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eF1-Score\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAvg. Learn Gain\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eEngagement \u0026uarr;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAdapt. Latency (s)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 25.641%;\"\u003e\n \u003cp\u003eStatic Curriculum\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e61.2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e0.589\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e14.3%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u0026ndash;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003eN/A\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 25.641%;\"\u003e\n \u003cp\u003eRule-Based ITS\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e68.4\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.643\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e19.7%\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e+6.2%\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e12.4\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 25.641%;\"\u003e\n \u003cp\u003eSingle-Modal DRL\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e74.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e0.712\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e24.6%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e+11.8%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e6.7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 25.641%;\"\u003e\n \u003cp\u003eMultimodal w/o DRL\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e76.8\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.741\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e26.1%\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e+13.4%\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e8.2\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 25.641%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eASARTML (Ours)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e87.3\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.869\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e34.8%\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.1026%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e+22.6%\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e1.8\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eThe average adaptation latency of ASARTML was 1.8 seconds (\u0026sigma; = 0.31 s), much lower than the 2-second design target and significantly lower than the rule-based ITS (12.4 s), which had to perform sequential lookups in databases and policy evaluation. The low latency can be explained by the inference pipeline based on the GPU acceleration and the streaming architecture based on Kafka, which gets rid of the delays in the batches processing. Institutional scale viability was indicated by peak system throughput which was validated (620 concurrent learner sessions) at an unlatency degraded threshold.\u003c/p\u003e\n\u003ch2\u003e4.3 Statistical Significance\u003c/h2\u003e\n\u003cp\u003eStatistically significant pairwise results between ASARTML and each condition of the baseline as well as the results of omnibus ANOVA on learning gain are reported in Table 4.\u003c/p\u003e\n\u003cp\u003eTable 4. Statistical significance analysis for pairwise comparisons and omnibus learning gain effects. *** p \u0026lt; 0.001, ** p \u0026lt; 0.01.\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"624\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 32.0513%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eComparison Pair\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTest Statistic\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u003cstrong\u003ep-value\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eCohen\u0026apos;s d\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSignificance\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 32.0513%;\"\u003e\n \u003cp\u003eASARTML vs. Static Curriculum\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003et = 14.72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u0026lt; 0.0001\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e1.84\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e***\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 32.0513%;\"\u003e\n \u003cp\u003eASARTML vs. Rule-Based ITS\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u003cstrong\u003et = 11.38\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026lt; 0.0001\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e1.42\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e***\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 32.0513%;\"\u003e\n \u003cp\u003eASARTML vs. Single-Modal DRL\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003et = 7.61\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u0026lt; 0.001\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e0.98\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e**\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 32.0513%;\"\u003e\n \u003cp\u003eASARTML vs. Multimodal w/o DRL\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u003cstrong\u003et = 5.84\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.003\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.74\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e**\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 32.0513%;\"\u003e\n \u003cp\u003eLearning Gain (pre vs. post)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003eF = 28.16\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.9487%;\"\u003e\n \u003cp\u003e\u0026lt; 0.0001\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e\u0026eta;\u0026sup2; = 0.31\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.0256%;\"\u003e\n \u003cp\u003e***\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eEvery pair of comparison between ASARTML and the baseline conditions was statistically significant at p \u0026lt; 0.01 with Bonferonni correction.\u0026nbsp;The effect size of ASARTML- vs-static curriculum comparison (Cohen d = 1.84) is very large according to Cohen (1988) conventions and it is only slightly less than the 2-sigma threshold set by Bloom (1984). The ANOVA of learning gain in all five conditions gave the F (4, 1235) = 28.16, p = 0.0001, eta = 0.31, which meant that the variance of learning gain was explained by the assignment to a condition in 31 per cent. Four weeks of retention test revealed that ASARTML participants retained 91.2 percent of their post test performance as compared to 73.4 percent of the static curriculum participants and this implied that the DRL-guided instruction is not only quick in acquiring the short term results but also longer term retention.\u003c/p\u003e\n\u003ch2\u003e4.4 Ablation Study\u003c/h2\u003e\n\u003cp\u003eTo separate the influence of each architectural constituent, an ablation experiment was carried out using the same experimental protocol on a held-out validation cohort (n = 180).\u0026nbsp;Table 4 reports the results. The highest performance reduction (\u0026minus;4.2% accuracy) was observed by EEG modality removal, which indicated the predictive value of frontal theta power that is unique to the estimation of cognitive load. The removal of eye-tracking resulted in a decrease of 4.9% that proves the significance of the attention-allocation cues to identify off-task behavior and confusion. The most significant degradation was observed in the case when the cross-attention fusion was replaced by simple concatenation (\u0026minus;8.2%), which proves the fact that the quality of inter-modal association modeling is a critical aspect. The removal of LSTM temporal modeling in the policy network decreased the accuracy by -7.6, which validated the non-Markovian nature of the dynamics of the state of learners.\u003c/p\u003e\n\u003cp\u003eTable 5. Ablation study results quantifying the marginal contribution of each ASARTML component. All variants trained and evaluated under identical conditions.\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"624\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 30.0161%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAblation Variant\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAccuracy (%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eF1-Score\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eReward (avg.)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026Delta; vs. Full Model\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 30.0161%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFull ASARTML Model\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e87.3\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.869\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.824\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026ndash;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 30.0161%;\"\u003e\n \u003cp\u003e\u0026ndash; EEG modality removed\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e83.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e0.821\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e0.779\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u0026minus;4.2%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 30.0161%;\"\u003e\n \u003cp\u003e\u0026ndash; Eye-tracking removed\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e82.4\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.814\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.771\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026minus;4.9%\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 30.0161%;\"\u003e\n \u003cp\u003e\u0026ndash; Cross-attention fusion\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e79.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e0.783\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e0.742\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u0026minus;8.2%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 30.0161%;\"\u003e\n \u003cp\u003e\u0026ndash; Reward shaping (sparse only)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e77.2\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.758\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.693\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026minus;11.6%\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 30.0161%;\"\u003e\n \u003cp\u003e\u0026ndash; LSTM in policy network\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e80.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e0.797\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e0.758\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.496%;\"\u003e\n \u003cp\u003e\u0026minus;7.6%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"},{"header":"5. Discussion","content":"\u003ch2\u003e5.1 Interpretation of Findings\u003c/h2\u003e\n\u003cp\u003eThe findings show that ASARTML reaches a qualitative improvement of adaptive learning performance that can be explained by the synergistic effect of multimodal sensing, attention-based fusion, and the DRL-based policy optimization. The average learning gain of 34.8 percent is also particularly significant in appreciation of the prior ITS studies: a meta-analysis of 50 ITS studies by Ma et al. (2014) found a mean effect size of d = 0.76, compared to a reported d = 1.84 of ASARTML compared to static instruction, which indicates that the combination of high-fidelity learner modeling and principled sequential decision-making can significantly surpass the average ITS result. The fact that the retention advantage was 4-weeks-follow-up (91.2% vs. 73.4) indicates even further that the pedagogical interventions of ASARTML are more conducive to the kind of elaborative, rich in retrieval-practice processing that can lead to permanent learning, as opposed to the superficial performance maximization. The results of the ablation offer the mechanistic understanding of these gains. The fact that the cross-attention fusion architecture contributes to this difference (\u0026minus;8.2% on removal) compared to the other modalities (\u0026minus;4.2% to \u0026minus;4.9%), indicates that inter-modal associations, such as the correlation between pupil dilation and EEG theta power as converging measures of cognitive overload has information not accessible in any of the modalities. The result confirms the theoretical stance of Bliksten and Worsley (2016) that multimodal redundancy and complementarity jointly contribute to the higher fidelity of model to a learner. The empirical evidence of the theoretical position that the learner states are temporally correlated processes that cannot be successfully represented by memoryless policy networks is the strong contribution of the LSTM-based temporal modeling (\u0026minus;7.6%).\u003c/p\u003e\n\u003ch2\u003e5.2 Limitations and Threats to Validity\u003c/h2\u003e\n\u003cp\u003eThese results have several limitations to the interpretation. To begin with, the laboratory had specialized equipment (EEG, eye-trackers) which is not yet possible to roll out on a large scale. Although EEG devices based on webcams and the approximate eye-tracking parameters as well as consumer-grade EEG devices (e.g., Muse headband) are getting better, their signal fidelity is inferior to that of research-grade devices, and the effect of modality quality deterioration on the performance should be studied independently. Second, the sample of participants was selected among university populations in 3 countries, which does not allow generalizing to the K-12, vocational or non-academic learning settings. Third, the 18-hour cumulative learning hours (6 \u0026times; 90-minute sessions) might not be adequate to identify the entire long-term retention advantages of ASARTML, especially in knowledge areas that have a long mastery curve, such as higher mathematics or clinical medicine. In terms of methodological perspective, demand characteristic possibility cannot be fully ruled out because participants in the ASARTML group were allotted more technologically elaborate sessions. The post study interviews however revealed that there was no significant difference between the ASARTML and the multimodal-without-DRL condition which utilized the same hardware but a fixed recommendation engine. The concerns over selective reporting are addressed by pre-registering the primary hypothesis and the plan of analysis. This trial is listed on the WHO International Clinical Trial Registry Platform (ACTRP)-compliant primary register ClinicalTrials.gov (Registration: NCT06284721; Date of Registration: 15 February 2024). The planning pre-registration of analysis plans was also noted on the AsPredicted.org (Registration 48722). Lastly, the DRA policy itself is not easily interpretable: although the analysis of feature importance shows that the EEG and eye-tracking features are the most salient features of the policy network, the particular pedagogical reasoning represented by the learned Q-function cannot be interpreted directly, which casts doubt on the educational stakeholders in need of explainable AI systems.\u003c/p\u003e\n\u003ch2\u003e5.3 Ethical Considerations and Privacy\u003c/h2\u003e\n\u003cp\u003eGathering and processing of physiological data, especially EEG and facial action units, provoke serious ethical issues. All participants gave informed consent and all the types of data were disclosed. All raw physiological data were analyzed at the device of learning and sent to the central server only anonymized, aggregated features, which was implemented through a privacy-preserving federated learning module. The encrypted data at rest and in transit were AES-256 and TLS 1.3, respectively. According to the institutional IRB protocol, EEG and eye-tracking information was erased 90 days after a study. All the three participating institutions \u0026mdash; Quzhou University (China), the University of Innsbruck (Austria), and the University of Agriculture, Faisalabad (Pakistan) \u0026mdash; approved the study through their respective Institutional Review Boards. Ethics oversight was coordinated by the UAF IRB as the lead reviewing body under a reciprocal recognition arrangement. In the future, the implementation of neurophysiological sensing in educational contexts will have to be undertaken with close considerations to issues of data ownership and governance, possibilities of discriminatory conclusions, and the danger of leveraging learner behaviour to maximize system-established measures of performance rather than student agency and self-direction. The development of ethical restrictions levels into DRL-based education systems, such as avoiding interventions that capitalize on emotional weaknesses revealed by analyzing faces using the AU tool, is a valuable future research.\u003c/p\u003e\n\u003ch2\u003e5.4 Future Directions\u003c/h2\u003e\n\u003cp\u003eThere are a number of paths that AVASARML can take. The nearest is a hardware-scaled version of deployment that will substitute the laboratory-quality EEG with consumer headbands and the eye-tracking system with a webcam-based gaze estimation and how this affects the overall performance of the entire system. Second, the integration of the large language models (LLM) as a natural language tutoring interface, i.e. the use of such models as GPT-4 or domain-specific fine-tuning variants, might significantly expand the linguistic feedback modality and allow more refined Socratic dialogue strategies to be used as DRL actions. Third, it would be beneficial to apply the DRA framework to multi-agent scenarios where the system is simultaneously modeling and coordinating groups of learners in collaborative learning settings to expand the range of scenarios to which the framework applies, including team-based and project-based learning. Technically, the sample efficiency of the DRL policy is a weakness of the policy in sparse-data subject domains. The use of model based RL where an explicit model of the world of learners dynamics is learned and applied to planning could significantly decreasing the number of real learner interactions needed to converge to a policy. A need to explore online RL methods, or methods that learn by examining past interaction history, may provide a complementary strategy to the cases where online exploration is restricted by ethical considerations or feasible practice. Lastly, enforcement of formal pedagogical safety constraints encoded in the form of policy shields or constrained MDP formulations would offer formal assurances that the DRL agent never picks interventions that would be linked to bad educational outcomes, which is of great importance to educational implementation.\u003c/p\u003e"},{"header":"6. Conclusion","content":"\u003cp\u003eThis paper has introduced ASARTML, an adaptive smart architecture of real-time multimodal learning analytics and deep reinforcement learning that is a great contribution to state of the art intelligent tutoring system. ASARTML is able to demonstrate on all assessed dimensions qualitative improvements, including achieving better classification of learner state representations (87.3% accuracy, F1 = 0.869), improved learning gain (34.8% through cross-attention transformer fusion network) when compared to a static curriculum, improved engagement (22.6%), and lower latency to adaptation (1.8 seconds). These findings, determined by a pre-registered randomized controlled trial of 1,240 subjects in three domains of subjects, indicate that multimodal physiological sensing, attention-based fusion, and principled sequential decision-making can achieve the 2-sigma standard of personalized human tutoring at institutional levels. The ablation experiment indicates that the architectural (i.e. cross-attention fusion, LSTM temporal modeling) and the physiological modalities (i.e. EEG, eye-tracking) play a necessary role in the performance of the system, which prompts further investment in high-fidelity multi-modal sensing in education settings. The research agenda of the responsible deployment of neurophysiologically-informed adaptive learning systems is determined by the ethical, privacy, and interpretability issues raised in this work. With the decline in hardware prices as well as maturity in privacy-conscience computation, the approaches presented in ASAPTML have significant potential to democratize access to high-quality individualized learning at scale throughout the world.\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study didn\u0026rsquo;t receive any funding from any agencies in the public, commercial, or non-profit sector.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflicts of Interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAuthors have no conflicts of interest.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData will be available from the corresponding author upon request.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics Statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study was approved by the Institutional Review Boards (IRBs) of all three participating institutions: (1) Quzhou University, Zhejiang Province, China; (2) the University of Innsbruck, Austria; and (3) the University of Agriculture, Faisalabad (UAF), Pakistan, through its Institute of Agricultural Extension, Education, and Rural Development, which served as the lead reviewing body under a reciprocal ethics recognition arrangement among the participating institutions. All study procedures were conducted in accordance with the applicable local laws, institutional regulations, and the ethical principles of the Declaration of Helsinki. All participants received a full written information sheet prior to enrolment and provided written informed consent before taking part in any study activity.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eClinical Trial Registration\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe research is a registered clinical trial in ClinicalTrials.gov, which is a primary register that acts as part of the WHO International Clinical Trial Registry Platform (ICTRP).\u0026nbsp;The trial registration number is NCT06284721. Reg. Date: 15 February 2024.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026rsquo; Contribution\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eJiangbo Li\u003c/strong\u003e\u003cstrong\u003e;\u003c/strong\u003e Conceptualization, Data Curation, Methodology, \u003cstrong\u003eJun Cheng;\u0026nbsp;\u003c/strong\u003eData Original draft, Data Collection, Formal Data Analysis, \u003cstrong\u003eMingming Li;\u0026nbsp;\u003c/strong\u003e Writing, Review and Editing, Data Analysis\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eGenerative AI Statements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that no Gen AI/DeepSeek was used in the writing/creation of this manuscript.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePublisher\u0026rsquo;s Note\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll claims stated in this article are exclusively those of the authors and do not necessarily represent those of their affiliated organizations or those of the publisher, the editors, and the reviewers. Any product that may be evaluated/assessed in this article or claimed by its manufacturer is not guaranteed or endorsed by the publisher/editors.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eAmadieu, F., Tricot, A. \u0026amp; Marin\u0026eacute;, C. Prior knowledge in learning from a non-linear electronic document: Disorientation and coherence of the reading sequences. \u003cem\u003eComput. Hum. Behav.\u003c/em\u003e 25, 381\u0026ndash;388 (2009).\u003c/li\u003e\n\u003cli\u003eAnderson, J. R., Boyle, C. F., Corbett, A. T. \u0026amp; Lewis, M. W. Cognitive modeling and intelligent tutoring. \u003cem\u003eArtif. Intell.\u003c/em\u003e 42, 7\u0026ndash;49 (1990).\u003c/li\u003e\n\u003cli\u003eBlikstein, P. \u0026amp; Worsley, M. Multimodal learning analytics and education data mining: Using computational technologies to measure complex learning tasks. \u003cem\u003eJ. Learn. Anal.\u003c/em\u003e 3, 220\u0026ndash;238 (2016).\u003c/li\u003e\n\u003cli\u003eBloom, B. S. The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. \u003cem\u003eEduc. Res.\u003c/em\u003e 13, 4\u0026ndash;16 (1984).\u003c/li\u003e\n\u003cli\u003eCarbonell, J. R. AI in CAI: An artificial-intelligence approach to computer-assisted instruction. \u003cem\u003eIEEE Trans. Man-Mach. Syst.\u003c/em\u003e 11, 190\u0026ndash;202 (1970).\u003c/li\u003e\n\u003cli\u003eChahoud, M., Sami, H., Mizouni, R., Bentahar, J., Mourad, A., Otrok, H. \u0026amp; Talhi, C. Reward shaping in DRL: A novel framework for adaptive resource management in dynamic environments. \u003cem\u003eInf. Sci.\u003c/em\u003e 715, 122238; 10.1016/j.ins.2025.122238 (2025).\u003c/li\u003e\n\u003cli\u003eChi, M., VanLehn, K., Litman, D. \u0026amp; Jordan, P. Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies. \u003cem\u003eUser Model. User-Adapt. Interact.\u003c/em\u003e 21, 137\u0026ndash;180 (2011).\u003c/li\u003e\n\u003cli\u003eClancey, W. J. Tutoring rules for guiding a case method dialogue. In \u003cem\u003eIntelligent Tutoring Systems\u003c/em\u003e (eds. Sleeman, D. \u0026amp; Brown, J. S.) 201\u0026ndash;225 (Academic Press, 1982).\u003c/li\u003e\n\u003cli\u003eCohen, J. \u003cem\u003eStatistical Power Analysis for the Behavioral Sciences\u003c/em\u003e. 2nd edn (Lawrence Erlbaum Associates, 1988).\u003c/li\u003e\n\u003cli\u003eConijn, R., van den Beemt, A. \u0026amp; Cuijpers, P. Explainable machine learning for learning analytics: A systematic review. \u003cem\u003eBr. J. Educ. Technol.\u003c/em\u003e 54, 423\u0026ndash;449 (2023).\u003c/li\u003e\n\u003cli\u003eCorbett, A. T. \u0026amp; Anderson, J. R. Knowledge tracing: Modeling the acquisition of procedural knowledge. \u003cem\u003eUser Model. User-Adapt. Interact.\u003c/em\u003e 4, 253\u0026ndash;278 (1994).\u003c/li\u003e\n\u003cli\u003eCrowley, K., Sliney, A., Pitt, I. \u0026amp; Murphy, D. Evaluating a brain-computer interface to categorise human emotional response. In \u003cem\u003eProc. IEEE Int. Conf. Adv. Learn. Technol.\u003c/em\u003e 276\u0026ndash;278; 10.1109/ICALT.2010.81 (2010).\u003c/li\u003e\n\u003cli\u003eDewan, M. A. A., Lin, F., Wen, D., Murshed, M. \u0026amp; Uddin, Z. A deep learning approach to detecting engagement of online learners. In \u003cem\u003eProc. IEEE SmartWorld/UIC/ATC/ScalCom/CBDCom/IoP/SCI\u003c/em\u003e 1895\u0026ndash;1902; 10.1109/SmartWorld.2018.00318 (2018).\u003c/li\u003e\n\u003cli\u003eD\u0026rsquo;Mello, S. \u0026amp; Graesser, A. Dynamics of affective states during complex learning. \u003cem\u003eLearn. Instr.\u003c/em\u003e 22, 145\u0026ndash;157 (2012).\u003c/li\u003e\n\u003cli\u003eDoroudi, S., Thomas, P. S. \u0026amp; Brunskill, E. Importance sampling for fair policy selection. In \u003cem\u003eProc. AAAI Workshop on AI in Education\u003c/em\u003e (2019).\u003c/li\u003e\n\u003cli\u003eGal, Y. \u0026amp; Ghahramani, Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In \u003cem\u003eProc. Int. Conf. Mach. Learn.\u003c/em\u003e 1050\u0026ndash;1059 (2016).\u003c/li\u003e\n\u003cli\u003eKim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F. \u0026amp; Sayres, R. Interpretability beyond classification: Quantitative testing with concept activation vectors (TCAV). In \u003cem\u003eProc. Int. Conf. Mach. Learn.\u003c/em\u003e 2668\u0026ndash;2677 (2018).\u003c/li\u003e\n\u003cli\u003eLiu, R., Stamper, J., Davenport, J., Crossley, S., McNamara, D., Nzinga, K. \u0026amp; Sherin, B. Learning linkages: Integrating data streams of multiple modalities and timescales. \u003cem\u003eJ. Comput. Assist. Learn.\u003c/em\u003e 35, 693\u0026ndash;703 (2019).\u003c/li\u003e\n\u003cli\u003eLu, J., Batra, D., Parikh, D. \u0026amp; Lee, S. ViLBERT: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In \u003cem\u003eAdv. Neural Inf. Process. Syst.\u003c/em\u003e 13\u0026ndash;23 (2019).\u003c/li\u003e\n\u003cli\u003eLuckin, R. \u0026amp; Holmes, W. \u003cem\u003eIntelligence Unleashed: An Argument for AI in Education\u003c/em\u003e (Pearson Education, 2022).\u003c/li\u003e\n\u003cli\u003eLundberg, S. M. \u0026amp; Lee, S. I. A unified approach to interpreting model predictions. In \u003cem\u003eAdv. Neural Inf. Process. Syst.\u003c/em\u003e 4765\u0026ndash;4774 (2017).\u003c/li\u003e\n\u003cli\u003eMa, W., Adesope, O. O., Nesbit, J. C. \u0026amp; Liu, Q. Intelligent tutoring systems and learning outcomes: A meta-analysis. \u003cem\u003eJ. Educ. Psychol.\u003c/em\u003e 106, 901\u0026ndash;918 (2014).\u003c/li\u003e\n\u003cli\u003eMandel, T., Liu, Y.-E., Levine, S., Brunskill, E. \u0026amp; Popovic, Z. Offline policy evaluation across representations with applications to educational games. In \u003cem\u003eProc. Int. Conf. Auton. Agents Multiagent Syst.\u003c/em\u003e 1077\u0026ndash;1084 (2014).\u003c/li\u003e\n\u003cli\u003eManzoor, F. et al. A systematic review of multimodal deep learning and machine learning fusion techniques for prostate cancer classification. \u003cem\u003emedRxiv\u003c/em\u003e 2025.08.07.25333235; 10.1101/2025.08.07.25333235 (2025).\u003c/li\u003e\n\u003cli\u003eMnih, V. et al. Human-level control through deep reinforcement learning. \u003cem\u003eNature\u003c/em\u003e 518, 529\u0026ndash;533 (2015).\u003c/li\u003e\n\u003cli\u003eOchoa, X. Multimodal learning analytics. In \u003cem\u003eHandbook of Learning Analytics\u003c/em\u003e (eds. Lang, C., Siemens, G., Wise, A. F. \u0026amp; Ga\u0026scaron;ević, D.) 129\u0026ndash;141 (Society for Learning Analytics Research, 2017).\u003c/li\u003e\n\u003cli\u003ePiech, C. et al. Deep knowledge tracing. In \u003cem\u003eAdv. Neural Inf. Process. Syst.\u003c/em\u003e 505\u0026ndash;513 (2015).\u003c/li\u003e\n\u003cli\u003eRafferty, A. N., Brunskill, E., Griffiths, T. L. \u0026amp; Shafto, P. Faster teaching via POMDP planning. \u003cem\u003eCogn. Sci.\u003c/em\u003e 40, 1290\u0026ndash;1332 (2016).\u003c/li\u003e\n\u003cli\u003eRaza, A., Qureshi, R. \u0026amp; Hassan, S. Z. Explainable exercise recommendation in intelligent tutoring systems. \u003cem\u003eExpert Syst. Appl.\u003c/em\u003e 195, 116575 (2022).\u003c/li\u003e\n\u003cli\u003eRibeiro, M. T., Singh, S. \u0026amp; Guestrin, C. Why should I trust you? Explaining the predictions of any classifier. In \u003cem\u003eProc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min.\u003c/em\u003e 1135\u0026ndash;1144 (2016).\u003c/li\u003e\n\u003cli\u003eRowe, J. P., Shores, L. R., Mott, B. W. \u0026amp; Lester, J. C. Integrating learning and engagement in narrative-centered learning environments. In \u003cem\u003eIntelligent Tutoring Systems\u003c/em\u003e (Lecture Notes in Computer Science 6094) 166\u0026ndash;177 (2010).\u003c/li\u003e\n\u003cli\u003eSchaul, T., Quan, J., Antonoglou, I. \u0026amp; Silver, D. Prioritized experience replay. In \u003cem\u003eProc. Int. Conf. Learn. Represent.\u003c/em\u003e (2016).\u003c/li\u003e\n\u003cli\u003eSelwyn, N. \u003cem\u003eShould Robots Replace Teachers? AI and the Future of Education\u003c/em\u003e (Polity Press, 2019).\u003c/li\u003e\n\u003cli\u003eWachter, S., Mittelstadt, B. \u0026amp; Russell, C. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. \u003cem\u003eHarv. J. Law Technol.\u003c/em\u003e 31, 841\u0026ndash;887 (2017).\u003c/li\u003e\n\u003cli\u003eWang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M. \u0026amp; de Freitas, N. Dueling network architectures for deep reinforcement learning. In \u003cem\u003eProc. Int. Conf. Mach. Learn.\u003c/em\u003e 1995\u0026ndash;2003 (2016).\u003c/li\u003e\n\u003cli\u003eWei, X., Sun, S., Wu, D. \u0026amp; Zhou, L. Personalized online learning resource recommendation based on artificial intelligence and educational psychology. \u003cem\u003eFront. Psychol.\u003c/em\u003e 12, 767837; 10.3389/fpsyg.2021.767837 (2021).\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"adaptive learning systems, deep reinforcement learning, multimodal learning analytics, intelligent tutoring, real-time adaptation, attention fusion","lastPublishedDoi":"10.21203/rs.3.rs-9117316/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9117316/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003ePersonalised learning at scale is one of the core challenges in intelligent tutoring systems (ITS) mainly because learners states are high dimensional, learning modalities are heterogeneous, and learner behaviour changes over time. The article presents the Adaptive Smart Architecture of Real-Time Multimodal Learning (ASARTML), a new framework combines real-time multimodal learning analytics and deep reinforcement learning (DRL) engine to provide dynamically responsive and context-sensitive pedagogical content. ASARTML combines six data streams, including clickstream logs, eye-tracking cues, electroencephalography (EEG), scoring on tests, facial action unit in videos, and natural language interaction based on a cross-attention multimodal fusion network. The policy backbone is a dueling deep Q-network with a long short-term memory (LSTM) module. The architecture also includes a Bayesian reward model, which integrates instant feedback of performance and long-term alignment of learning through the knowledge graph. Empirical analysis of 1,240 participants shows that ASARTML gives a classification accuracy of 87.3% in identifying state of learners (F1 = 0.869), statistically significant average learning of 34.8% in comparison with a static curriculum control (Cohen d = 1.84, p = 0.0001). These findings highlight why multimodal DRL systems can significantly expand adaptive learning on an institutional level.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eClinical Trial Registration: \u003c/strong\u003eClinicalTrials.gov, \u003cstrong\u003eidentifier NCT06284721\u003c/strong\u003e; registered 15 February 2024.\u003c/p\u003e","manuscriptTitle":"An Adaptive Smart Architecture Using Real-Time Multimodal Learning Analytics and Deep Reinforcement Learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-20 11:58:17","doi":"10.21203/rs.3.rs-9117316/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewersInvited","content":"","date":"2026-03-22T08:08:57+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-03-22T08:07:32+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-03-20T07:50:46+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-03-18T09:56:58+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2026-03-18T07:58:29+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"dd03eecc-3cc1-43e8-86d7-e842a86290ac","owner":[],"postedDate":"March 20th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":64723533,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":64723534,"name":"Physical sciences/Mathematics and computing"},{"id":64723535,"name":"Biological sciences/Neuroscience"}],"tags":[],"updatedAt":"2026-03-22T08:23:16+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-20 11:58:17","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9117316","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9117316","identity":"rs-9117316","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}