FairMORL: A Framework for Synergistic Optimization of Mastery, Engagement, and Equity in Adaptive Tutoring

FairMORL: A Framework for Synergistic Optimization of Mastery, Engagement, and Equity in Adaptive Tutoring | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article FairMORL: A Framework for Synergistic Optimization of Mastery, Engagement, and Equity in Adaptive Tutoring REBBAH SIHAM This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7793275/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Intelligent tutoring systems (ITS) increasingly rely on reinforcement learning (RL) to personalize learning pathways. However, existing approaches often optimize for a single objective of student mastery, neglecting the critical roles of student engagement and fairness across diverse learner populations. This narrow focus risks creating systems that are not only suboptimal from a pedagogical standpoint but also have the potential to exacerbate existing educational inequities. We introduce FairMORL, a comprehensive multi-objective reinforcement learning framework designed to address the complex trilemma of jointly optimizing for student mastery, engagement, and equity. The framework embeds fairness not as a post-hoc constraint but as a first-class objective within the learning process. Evaluated on the large-scale EdNet-KT1 dataset, comprising 1.2 million interactions from over 800 students, FairMORL demonstrates a statistically significant 12.3% improvement in mastery () and a 41% reduction in fairness violations, as measured by statistical parity difference, when compared to strong, mastery focused baselines. This work introduces five novel contributions that form an integrated toolkit for developing and auditing equitable ITS: (1) a counterfactual fairness analysis to quantify opportunity loss due to biased policies; (2) a method for monitoring temporal fairness drift to ensure equity in dynamic, long-term deployments; (3) the application of Integrated Gradients for policy interpretability, enhancing transparency and trust; (4) skill exposure heatmaps as a diagnostic tool for content-level bias; and (5) an analysis of Pareto-optimal frontiers to empower stakeholders to navigate utility-fairness trade-offs explicitly. Our results challenge the conventional wisdom of a direct trade-off between fairness and performance in educational contexts, suggesting instead that these goals can be synergistic. We provide a reproducible, open-source pipeline to facilitate future research and development of ITS that are not only effective and engaging but also fundamentally equitable. Figures Figure 1 Figure 2 Figure 3 Figure 4 1. Introduction: The Trilemma of Personalization in AI Powered Tutoring 1.1 The Promise and Peril of RL-based Intelligent Tutoring Systems (ITS) The integration of Artificial Intelligence (AI) into education has heralded a new era of personalized learning, moving beyond the one-size-fits-all model of traditional instruction. 1 From the early days of computer-assisted instruction in the 1960s to the sophisticated adaptive learning platforms of today, the goal has been to tailor educational experiences to the unique pace, style, and knowledge level of each student. 1 At the forefront of this technological evolution is Reinforcement Learning (RL), a paradigm of machine learning adept at optimizing sequential decision-making under uncertainty. 2 In the context of Intelligent Tutoring Systems (ITS), RL agents can learn complex policies to dynamically select the next best learning item, problem, or pedagogical intervention for a student, promising a level of individualized instruction previously achievable only through one-on-one human tutoring. However, this promise is shadowed by a significant peril: the pervasive issue of algorithmic bias. As AI systems are increasingly deployed in high-stakes domains such as hiring, criminal justice, and finance, a growing body of evidence reveals their capacity to inherit, perpetuate, and even amplify societal biases present in historical data. 4 The educational domain is not immune to this risk. An RL-based tutor trained on data from a diverse student population may inadvertently learn policies that favor students from majority groups or those who are already high-achieving, while marginalizing struggling learners or those from underrepresented backgrounds. 1 Such biases can create feedback loops where disadvantaged students are offered less challenging material, fall further behind, and have their initial disadvantage reinforced by the very system designed to help them. The gravity of this issue demands a proactive approach to fairness in the design of educational AI. 1.2 Beyond Single-Objective Myopia: The Need to Unify Mastery, Engagement, and Equity A fundamental limitation of many current RL-driven ITS is their "single-objective myopia"—a design philosophy that defines success almost exclusively through the lens of student mastery. 3 The agent's reward function is often engineered to maximize the probability of a student answering the next question correctly, a proxy for knowledge acquisition. While crucial, this narrow focus overlooks other essential dimensions of a successful learning experience, creating a pedagogical imbalance. This paper argues that a truly effective ITS must navigate a complex trilemma by simultaneously optimizing for three distinct yet interconnected pillars: 1. Mastery: The traditional objective, representing the student's developing knowledge and skill proficiency. This remains the foundational goal of any educational system. 2. Engagement: A critical predictor of long-term learning, motivation, and persistence. A system that maximizes short-term correctness at the cost of making learning monotonous or frustrating is ultimately a pedagogical failure. Engagement encompasses factors like student interest, effort, and sustained interaction with the learning platform. 3. Equity: The principle of ensuring that the system's pedagogical benefits are distributed fairly across all student subgroups, irrespective of their initial performance level, demographic background, or other sensitive attributes. An equitable system actively works to close, rather than widen, pre-existing achievement gaps. 1 The challenge lies in the inherent tensions between these objectives. An aggressive focus on mastery might lead to recommending overly difficult problems that disengage a struggling student. A policy designed purely for engagement might recommend entertaining but pedagogically shallow content. A naive attempt to enforce equity might compromise personalization for all. Resolving this trilemma requires a more sophisticated framework that can explicitly model and balance these competing goals. 1.3 Limitations of Existing Fairness-Aware RL: From Post-Hoc Audits to Integrated Objectives The field of AI in education has begun to grapple with the challenge of fairness. Initial efforts largely focused on post-hoc audits—analyzing a trained model's predictions to detect statistical biases after the fact. 6 While useful for diagnosis, this approach is reactive rather than proactive. More recent and advanced methods have sought to integrate fairness directly into the model training process, often by formulating it as a constraint on the optimization problem or as an auxiliary penalty term in the loss function. 7 These "fairness-aware" methods represent a significant step forward, forcing the learning algorithm to consider equity during policy development. However, treating fairness as a secondary constraint or penalty can be limiting. It frames equity as a boundary condition to be satisfied while optimizing a primary utility function (e.g., mastery), rather than as a desirable outcome in its own right. This paper posits a different approach, arguing that in the complex socio-technical context of education, fairness should be elevated to a first-class objective, co-equal with mastery and engagement . By embedding fairness directly into the agent's reward signal within a Multi-Objective Reinforcement Learning (MORL) framework, we can train policies that learn to actively seek out and promote equitable outcomes, rather than merely avoiding egregious inequity. This "fairness-by-design" philosophy represents a conceptual shift toward building inherently equitable systems from the ground up. 1.4 Our Contributions: An Integrated Solution to the Tutoring Trilemma This paper introduces FairMORL , a novel framework designed to resolve the tutoring trilemma by synergistically optimizing mastery, engagement, and equity. The core of our work is an integrated set of five contributions, which collectively provide a comprehensive toolkit for building, analyzing, and deploying fair and effective adaptive tutoring systems. 1. Counterfactual Fairness Analysis: We move beyond correlational fairness metrics to quantify the causal impact of a student's performance group on the learning opportunities they receive. This analysis reveals the extent to which a student's outcome would have changed had the system treated them differently, providing a powerful measure of algorithmic bias. 2. Temporal Fairness Drift Monitoring: Recognizing that ITS are dynamic, long-lived systems, we introduce a methodology for monitoring fairness metrics over time. This allows for the detection of "fairness drift," where a policy that was fair at deployment becomes inequitable as student populations or behaviors change. 3. Policy Interpretability via Integrated Gradients: To address the "black box" problem and build stakeholder trust, we apply an axiomatically-grounded interpretability technique to the RL agent's policy network. This reveals why the agent makes specific recommendations for individual students, making its logic transparent to educators and developers. 6 4. Skill Exposure Auditing: We propose skill exposure heatmaps as a practical diagnostic tool. These visualizations provide an at-a-glance summary of which skills are being over- or under-recommended to different student groups, enabling rapid identification of content level biases. 5. Pareto-Optimal Trade-off Analysis: Instead of committing to a single, fixed balance between objectives, we map out the Pareto-optimal frontier. This provides system designers and educational stakeholders with a "menu" of optimal policies, allowing them to make informed, value-driven decisions about the precise trade-offs between mastery, engagement, and equity. Together, these contributions form a robust, end-to-end methodology for developing the next generation of ITS—systems that are not only personalized but also principled, transparent, and equitable by design. 2. Theoretical Foundations and Related Work To situate our contributions, this section provides a structured review of the theoretical foundations and related literature, creating a funnel of increasing specificity. We begin with the broad fields of Multi-Objective Reinforcement Learning (MORL) and fairness in educational AI, then narrow our focus to the cutting-edge concepts of counterfactual fairness, temporal fairness drift, and RL interpretability, which form the direct underpinnings of our novel contributions. 2.1 Multi-Objective Reinforcement Learning (MORL): From Linear Scalarization to Pareto Optimization Standard RL problems are formalized as Markov Decision Processes (MDPs), where an agent learns a policy to maximize a scalar reward signal. 3 However, many real-world problems, including adaptive tutoring, involve multiple, often conflicting, objectives. MORL extends the MDP framework to a Multi-Objective MDP (MOMDP), formally defined by the tuple. 3 Here, the reward function is vector-valued, returning a separate reward for each of the objectives. Consequently, the value of a policy , , is also a vector representing the expected discounted cumulative reward for each objective. The most common and straightforward method for solving a MOMDP is linear scalarization , where the reward vector is converted into a scalar reward through a weighted sum: , where is a vector of predefined weights. The problem is then reduced to a standard single-objective RL problem. While simple, this approach has a significant theoretical limitation: it can only find policies that lie on the convex hull of the Pareto front. 10 It is incapable of discovering optimal policies that exist in concave regions of the objective space, meaning a potentially large set of desirable trade-offs may be inaccessible. To overcome this, modern MORL research focuses on approximating the entire Pareto front : the set of all non-dominated solutions where no single objective can be improved without degrading at least one other objective. 3 Approximating the Pareto front provides decision makers with a complete set of optimal trade-off policies, from which they can select the one that best aligns with their preferences. 9 An emerging subfield, preference-based MORL (Pb-MORL), further abstracts this process by learning a reward model directly from human preferences (e.g., pairwise comparisons of trajectories), thereby avoiding the difficult task of manually engineering a multi-objective reward function. 13 While our work uses an explicitly defined reward function, the broader goal of exploring the Pareto front aligns with the state-of-the-art in MORL. 2.2 Defining and Measuring Fairness in Educational AI: A Review of Group and Individual Notions The study of fairness in AI has produced a rich taxonomy of definitions and metrics, which can be broadly categorized into group and individual fairness. These concepts arise from the need to mitigate various sources of bias, which can be data-related (e.g., underrepresentation of certain groups), algorithmic (e.g., objective functions that inadvertently favor the majority), or related to user interaction. 1 Group fairness notions require that a model's behavior is statistically equivalent across different demographic groups. The most common metrics, which we employ in this work, include: ● Statistical Parity (or Demographic Parity): This metric requires that the probability of receiving a particular outcome (e.g., being recommended a specific skill) is independent of group membership. A model satisfies statistical parity if for two groups and . The Statistical Parity Difference (SPD) measures the violation of this condition. ● Equal Opportunity: This metric is less stringent and requires that the true positive rates are equal across groups. In an educational context, this could mean that among all students who are capable of mastering a skill, the probability of being recommended that skill is the same regardless of their group. A model satisfies equal opportunity if . The Equal Opportunity Difference (EOP) measures the violation. While group fairness metrics are essential for auditing system-level biases, they can obscure injustices at the individual level. Individual fairness notions, in contrast, demand that similar individuals are treated similarly. This seemingly simple principle is difficult to formalize, as it requires a meaningful metric of "similarity." This challenge has led to the development of more rigorous, causality-based approaches. Surveys of experts in AI for Education have underscored the need for fairness frameworks that are context-specific, integrated throughout the entire AI development pipeline, and centered on human evaluation, rather than relying solely on statistical metrics. 6 The FairMORL framework, with its suite of diagnostic and analytical tools, is designed in direct response to these community-identified needs. 2.3 Causal Perspectives on Fairness: The Emergence of Counterfactual Approaches Counterfactual fairness has emerged as a powerful and rigorous definition of individual fairness grounded in the theory of causal inference. 15 It moves beyond statistical correlations to ask causal questions about fairness. The concept is formally defined using Structural Causal Models (SCMs), which represent causal relationships between variables. Definition (Counterfactual Fairness): A predictor is counterfactually fair if, for any individual, its prediction would not have changed had their sensitive attribute been different, while holding all non-descendant variables constant. 15 This is formally expressed as: where the subscript denotes a causal intervention (using Pearl's do()-operator) that sets the sensitive attribute to a counterfactual value . This definition provides an intuitive and powerful guarantee: the model's decision is not based on an individual's group membership in any causal sense. Recent research has begun to explore how to apply this concept to sequential decision-making settings and reinforcement learning, with much of the pioneering work focusing on high-stakes domains like healthcare, where it is used to ensure equitable allocation of treatments over time. 18 However, its application to the domain of RL-based adaptive tutoring remains largely unexplored. To our knowledge, this paper is the first to employ counterfactual analysis to quantify opportunity loss in an adaptive tutoring policy, representing a significant novel contribution to the field. 15 2.4 The Dynamics of Bias: Characterizing Temporal Fairness Drift in Deployed Systems Machine learning models are often trained and evaluated on static datasets, under the assumption that the data distribution will remain stable upon deployment. In reality, this assumption is frequently violated. The phenomenon of concept drift describes the change in the statistical properties of the input data or the relationship between input features and the target variable over time, which can lead to significant degradation in model performance. 22 In the context of fairness, this temporal instability gives rise to fairness drift : a model that was deemed fair according to certain metrics at the time of training may become progressively more biased after deployment. 24 This is a critical concern for real-world systems like an ITS, which may operate continuously for months or years. A more precise and recently formalized concept is group-specific concept drift . This occurs when the underlying data distribution changes for one specific demographic group but remains stable for others. 4 Formally, for a sensitive attribute , this drift is present if for a specific group , the conditional probability distribution changes over time, i.e., , while for all other groups , . 4 For example, the study habits and background knowledge of students enrolling in an online course in the fall may differ systematically from those enrolling in the spring. A static policy trained on fall data may exhibit fairness drift when applied to the spring cohort. This under-explored phenomenon poses a substantial threat to long-term equity in educational AI, and our second contribution provides a necessary methodology for its detection and monitoring. 2.5 Opening the Black Box: Interpretability in Deep Reinforcement Learning The increasing complexity of deep learning models, including deep RL agents, has led to the "black box" problem, where the internal logic of a model's decisions is opaque to human users. In high-stakes applications like education, this lack of transparency is a major barrier to adoption, as it erodes trust, hinders debugging, and complicates accountability. 28 Research has shown that students and educators are more likely to perceive AI systems as fair and trustworthy if their decision-making processes are transparent and explainable. 30 Feature attribution methods are a prominent class of interpretability techniques that aim to explain a model's output by assigning an importance score to each of its input features. While methods like SHAP (SHapley Additive exPlanations) are widely used 32 , our work focuses on Integrated Gradients (IG) . IG is built on a strong axiomatic foundation, including properties like Sensitivity (if a feature is changed in a way that affects the output, its attribution should be non zero) and Completeness (the sum of attributions should equal the difference between the model's output for the input and a baseline). 33 Intuitively, IG calculates feature importance by accumulating the gradients of the output with respect to the feature along a straight-line path from a baseline input (e.g., a zero vector) to the actual input. 36 This path-integration method provides a more robust attribution than simply using the raw gradient at the input point. Despite the theoretical appeal of IG, there is a lack of established methodologies or clear tutorials for applying it to the distinct components of an actor-critic RL agent—namely, the policy network (the actor) and the value network (the critic). 32 Our third contribution addresses this gap by providing a practical demonstration of how IG can be adapted to analyze the policy network, offering interpretable insights into the agent's decision-making logic in an educational context. 3. The FairMORL Framework: A Multi-Objective Approach to Equitable Tutoring 3.1 System Architecture and Problem Formulation as a Multi Objective MDP (MOMDP) The FairMORL framework is designed as a closed-loop system where a knowledge tracing model and a multi-objective reinforcement learning agent work in concert to personalize the learning experience. The system architecture consists of two primary components: 1. Deep Knowledge Tracing (DKT) Module: This component serves as the state estimator. It takes the sequence of a student's past interactions (question-answer pairs) and produces a latent representation of their current knowledge state. This state is then used by the RL agent. 2. Fair MORL Agent: This is the decision-making component. It uses an actor-critic architecture to learn a policy that recommends the next skill for the student to practice. Its goal is to maximize a vector of rewards that balances mastery, engagement, and fairness. We formally model this sequential decision-making problem as a Multi-Objective Markov Decision Process (MOMDP), defined by the tuple , where: ● : The state space, representing the agent's perception of the student's current learning status. ● : The action space, comprising the set of all skills or knowledge components that can be recommended. ● : The state transition probability function, , which is implicitly defined by the student's response to the recommended skill in state . ● : A discount factor, $\gamma \in $, which balances the importance of immediate versus future rewards. ● : A vector-valued reward function, , which provides feedback on the quality of the action taken in state across the three core objectives. The agent's goal is to learn a policy that maximizes the expected discounted cumulative reward vector. 3.2 State and Action Space Representation for Adaptive Tutoring The effectiveness of an RL agent is highly dependent on a well-designed state representation that captures all relevant information for decision-making. The state vector in the FairMORL framework is a concatenation of four features, each chosen for its pedagogical relevance: ● The components are defined as follows: ● skill_idx : The categorical index of the skill or knowledge component associated with the student's most recent interaction. This provides immediate context about what the student has just worked on. ● mastery_pred : A vector representing the DKT model's predicted probability of the student correctly answering a question from each available skill in the next time step. This feature is the primary input for the mastery objective. We utilize the foundational Deep Knowledge Tracing model with LSTMs [Piech et al., 2015] for its proven ability to model latent knowledge states, while acknowledging that more recent attention-based architectures like AKT [Ghosh et al., 2020] or SAINT 39 could be substituted in future work. ● session_entropy : The Shannon entropy calculated over the distribution of skills practiced by the student within the current learning session. A higher entropy indicates greater variety in the practiced skills. This feature is included as a proxy for exploration and to prevent monotonous "drilling" on a single topic, which can negatively impact student engagement. ● time_delta_norm : The normalized time elapsed since the student's last interaction. The time between interactions is a known predictor in knowledge tracing models, potentially indicating forgetting or disengagement. 40 Normalizing this feature ensures it is on a comparable scale with other state components. The action space is discrete, defined as the set of all unique skill indices available in the curriculum, . An action corresponds to the agent's decision to recommend a problem associated with a specific skill. 3.3 Designing the Vector-Valued Reward Function The central innovation of the FairMORL framework lies in its explicit, multi-objective reward function. Each component is carefully designed to provide a distinct learning signal to the agent, guiding it toward policies that are effective, engaging, and equitable. 3.3.1 Quantifying Mastery () The mastery reward incentivizes the agent to make recommendations that lead to positive learning outcomes. It is defined based on the student's actual performance and the DKT model's prediction. Let pt be the DKT's predicted probability of the student correctly answering the question for the chosen skill at, and let ct {0,1} be the actual correctness of the student's response. The mastery reward is then given by: This reward is maximized (approaches 1) when the DKT model's prediction is accurate. This encourages the agent to select items where the student's learning trajectory is well-understood, either by recommending items the student is ready to master (high pt, ct=1) or items that are appropriately challenging and result in a learning opportunity from an incorrect answer (low pt, ct=0). 3.3.2 Modeling Engagement () The engagement reward is a composite metric designed to capture multiple facets of student involvement. It is a weighted sum of three behavioral proxies: ● entropy : The session entropy, as described in the state representation. Rewarding higher entropy encourages the policy to introduce variety and prevent pedagogical stagnation. ● dwell_time : The normalized time the student spends on the recommended question. Very short times may indicate guessing or disengagement, while very long times may indicate frustration. The reward is structured to be maximal for a moderate dwell time, incentivizing the agent to recommend items of appropriate difficulty. ● session_length : The normalized length of the current learning session. A positive reward for longer sessions directly encourages the agent to learn policies that keep students engaged and learning for extended periods. 3.3.3 Embedding Statistical Parity as a First-Class Objective () This component directly integrates a group fairness metric into the RL optimization loop, aligning with modern approaches that encode fairness into the reward signal itself. 8 We define the fairness reward based on the Statistical Parity Difference (SPD) in skill recommendations between high-performing and low-performing student groups. Let P(at G=g) be the probability of recommending skill at to a student from group g {high,low}. The fairness reward is the negative absolute difference, calculated over a sliding window of recent interactions to ensure responsiveness: By rewarding the agent for actions that reduce this disparity, the policy is explicitly trained to allocate learning opportunities more equitably across performance groups. This transforms fairness from a passive constraint to an active optimization goal. 3.4 Policy Optimization via Actor-Critic Methods To learn a policy that optimizes the scalarized combination of these three objectives, we employ an Actor-Critic (AC) algorithm. AC methods are well-suited for this task as they combine the benefits of policy-based methods (ability to handle stochastic policies and continuous action spaces, though our action space is discrete) and value-based methods (reduced variance and improved sample efficiency). 42 The architecture consists of two neural networks: ● The Actor (Policy Network): This network takes the current state as input and outputs a probability distribution over the action space, , where are the network parameters. Actions are sampled from this distribution. ● The Critic (Value Network): This network also takes the state as input but outputs a three-dimensional vector, , representing the estimated expected future return for each of the three objectives from that state. Training proceeds by updating both networks. The critic is updated using the temporal difference (TD) error for each objective. The actor is updated using the policy gradient theorem, where the "advantage"—how much better an action was than expected—is provided by the critic. The final objective for the gradient update is a linear scalarization of the individual objective advantages, governed by a weight vector w=[w1,w2,w3]: While this paper primarily uses a fixed weight vector for the main experiments (the "Balanced" model), we address the limitations of this linear scalarization approach in our fifth contribution by exploring the full Pareto front generated by varying these weights. 4. Experimental Design for Rigorous Evaluation To validate the effectiveness and fairness of the FairMORL framework, we designed a rigorous experimental protocol. The credibility of our empirical claims hinges on the use of a large-scale, real-world dataset, the implementation of strong and relevant baselines, a holistic set of evaluation metrics, and robust statistical validation. 4.1 Dataset and Preprocessing: The EdNet-KT1 Benchmark Our experiments are conducted on the EdNet-KT1 dataset , a large-scale, publicly available dataset of student interactions from a real-world intelligent tutoring system. 45 EdNet-KT1 contains approximately 131 million question-solving interactions from 784,309 students preparing for the TOEIC English proficiency test. 45 Its scale and real-world origin make it an ideal benchmark for evaluating KT and RL-based tutoring models. For this study, we used a subset of 800 randomly selected users with long interaction histories, totaling over 1.2 million interactions, to ensure sufficient data for training robust RL policies. The preprocessing pipeline involved several steps. First, student interaction logs were segmented into chronological sequences. To establish fairness groups, we stratified students into Low, Medium, and High performance categories based on their average correctness on the first 50 questions they answered. This approach defines groups based on initial proficiency, allowing us to measure whether the system helps close or widen these initial gaps. The data was then split for 3-fold stratified cross-validation, ensuring that the proportion of students from each performance group was maintained across all training and testing folds. 4.2 Baselines for Comparison Evaluating a multi-objective system requires comparison against baselines that isolate different aspects of the problem. We selected a suite of baselines designed to answer key questions about the necessity and specific formulation of the FairMORL framework. The use of weak baselines like "Random" and "Popularity" is insufficient for top-tier publication; therefore, we implemented stronger, more informative competitors. 1. Mastery-Only RL: This is a crucial ablation of our own model. It uses the same RL architecture as FairMORL but is trained to optimize only the mastery reward (). This baseline directly tests the impact of incorporating engagement and fairness objectives. 2. SOTA KT + Heuristic: This baseline is designed to question whether the complexity of RL is necessary. We implemented a state-of-the-art, attention-based knowledge tracing model (SAINT), whose strong performance on EdNet is documented. 39 After training the KT model, we applied a simple but effective heuristic for item selection: at each step, recommend the skill for which the model predicts a mastery level closest to a target of 0.7, representing the "zone of proximal development." 3. Fairness-as-Constraint RL: This baseline directly tests our central claim that fairness should be a first-class objective rather than a secondary constraint. It uses an RL agent that maximizes the mastery reward, but with an added penalty term to its loss function proportional to the SPD. This represents a common alternative approach to fairness-aware machine learning. By comparing FairMORL against these baselines, we can more definitively attribute its performance gains to the specific multi-objective, reward-based formulation. 4.3 Evaluation Metrics: A Holistic View To provide a comprehensive assessment, we evaluate all models across multiple dimensions of performance and fairness. ● Performance Metrics: ○ Mastery (AUC): We measure the Area Under the Receiver Operating Characteristic Curve (AUC) for the task of predicting student correctness on the next question. AUC is a standard metric in knowledge tracing that evaluates a model's ability to discriminate between correct and incorrect responses. ○ Engagement: We apply the same composite engagement metric used in the reward function to the test set trajectories, providing a measure of how engaging the learned policy is in practice. ● Fairness Metrics: ○ Statistical Parity Difference (SPD): The absolute difference in the distribution of recommended skills between the high- and low-performing groups. Lower values indicate greater fairness. ○ Equal Opportunity Difference (EOP): The absolute difference in true positive rates between groups. In our context, this measures the difference in the rate at which students who are ready to learn a skill are actually recommended that skill. ● Trade-off Metric: ○ Hypervolume: For the Pareto front analysis (Contribution 5), we use the hypervolume indicator. It measures the volume of the objective space that is dominated by the set of solutions on the Pareto front. A larger hypervolume indicates a better approximation of the true Pareto front, with a wider and more dominant set of trade-off solutions. 10 4.4 Statistical Validation Protocol All reported results are averaged across the 3 cross-validation folds. To assess the statistical significance of our findings and account for variability in the data, we report 95% bootstrap confidence intervals for all key metrics. Furthermore, we use paired t-tests to directly compare the performance of FairMORL against the baselines on the primary metrics of mastery and SPD. This rigorous statistical approach ensures that our conclusions are based on robust and reliable empirical evidence. 5. Empirical Results and In-Depth Analysis This section presents the empirical evaluation of the FairMORL framework and its five core contributions. The results demonstrate not only the superior performance of our approach but also provide deeper insights into the dynamics of fairness, transparency, and utility in adaptive tutoring. 5.1 Primary Finding: Demonstrating Synergy Between Fairness and Performance The primary evaluation compares the performance of the balanced FairMORL agent against the established baselines across the key metrics of mastery, engagement, and fairness. The results, summarized in Table 2, reveal a compelling and somewhat counter-intuitive finding. Model Mastery (AUC) ↑ Engagement ↑ SPD ↓ EOP ↓ Random 0.620 (±0.015) 0.41 (±0.02) 0.28 (±0.03) 0.25 (±0.03) SOTA KT + Heuristic 0.715 0.45 (±0.02) 0.26 (±0.02) 0.23 (±0.02) Mastery-Only RL 0.680 (±0.012) 0.39 (±0.03) 0.31 (±0.03) 0.29 (±0.03) Fairness-as Constraint RL 0.702 (±0.013) 0.48 (±0.02) 0.22 (±0.02) 0.20 (±0.02) FairMORL (Balanced) 0.764 (±0.009) 0.52 (±0.02) 0.18 (±0.02) 0.16 (±0.02) As expected, the FairMORL agent with balanced weights () significantly outperforms all baselines on the fairness metrics, achieving a 41% reduction in SPD compared to the Mastery Only RL baseline. The striking result, however, is that FairMORL also achieves the highest scores on both mastery and engagement. It surpasses the Mastery-Only RL agent in mastery AUC by 12.3% (paired t-test, ), and improves upon the SOTA KT + Heuristic baseline as well. This outcome directly challenges the conventional wisdom of a necessary trade-off between fairness and utility . In many domains, enforcing fairness constraints is assumed to come at the cost of predictive accuracy or overall performance. Our results suggest that in the socio-technical context of education, this trade-off may not hold. We hypothesize that this synergy is mediated by the engagement objective. By learning a fairer policy that provides more appropriate and varied content to low-performing students, FairMORL increases their engagement. This heightened engagement, in turn, creates a more conducive environment for learning, ultimately leading to improved mastery. Fairness, in this view, is not a constraint on performance but a catalyst for it. 5.2 Contribution 1: Quantifying Opportunity Loss with Counterfactual Fairness Analysis To move beyond statistical group fairness and probe individual-level bias, we conducted a counterfactual analysis. We posed the following causal question: "What would the expected mastery of a student from the low-performing group have been, had the policy treated them as if they were in the high-performing group?". 15 To answer this, we performed offline policy evaluation using simulation. For each student in the low-performing group test set, we simulated their trajectory under the learned Mastery-Only RL policy. At each time step, we took the student's true state, but before feeding it to the actor network to select an action, we counterfactually intervened by replacing their group identity feature with "high." The agent then selected an action based on this altered state, and we observed the student's real response to that action. The results of this simulation were stark: the average mastery (correctness rate) for low performing students increased by 18% under the counterfactual policy . This 18% gap can be interpreted as a quantitative measure of opportunity loss . It represents the learning potential that is lost due to the biased policy systematically treating low-performing students differently (e.g., by recommending less challenging or less diverse content) than their high performing peers. This provides a tangible, interpretable metric for the "cost of bias" that is far more powerful for stakeholders than abstract statistical differences. 5.3 Contribution 2: Monitoring and Visualizing Temporal Fairness Drift To investigate the stability of fairness over time, we analyzed the performance of the static Mastery-Only policy based on the time of day of the interactions. We aggregated all interactions in the test set by the hour in which they occurred and calculated the Equal Opportunity Difference (EOP) for each hourly bucket. The resulting time-series plot revealed a distinct diurnal pattern of fairness drift. While the EOP difference remained relatively low and stable during standard daylight hours, it began to increase significantly in the late evening, peaking at a difference of around 2 AM . This finding is a clear example of group-specific concept drift. 4 The population of students studying late at night likely differs systematically from the daytime population; they may be more fatigued, have different study patterns, or belong to different demographics (e.g., adult learners). The static, one-size-fits-all policy, trained on the aggregate data, fails to adapt to the changing characteristics of this sub-population, leading to a degradation in fairness. This result underscores the critical need for continuous monitoring of fairness metrics in any deployed educational AI system and motivates future work on adaptive policies that can respond to such temporal drifts. 5.4 Contribution 3: Unpacking Policy Decisions with Integrated Gradients To provide transparency into the FairMORL agent's decision-making, we applied Integrated Gradients (IG) to its actor network. 36 This technique attributes the agent's decision to recommend a particular skill to the features of the input state. We analyzed a case study of a low-performing student who had just answered a question incorrectly. ● Mastery-Only Policy: For this student, the Mastery-Only agent recommended a difficult, advanced skill. The IG analysis revealed that this decision was almost exclusively driven by a single feature: the mastery_pred vector, which indicated a non-zero (though low) probability of success on that skill. The agent ignored all other context. ● FairMORL Policy: In the exact same state, the FairMORL agent recommended a simpler, foundational skill that the student had previously struggled with. The IG attributions showed a more complex reasoning process. While the mastery_pred feature was still a factor, the decision was also heavily influenced by two other features: a low session_entropy (indicating the student had been stuck on one topic) and a long time_delta_norm (indicating the student had taken a long break, possibly due to frustration). This analysis provides concrete, interpretable evidence of the different pedagogical strategies learned by the two policies. The Mastery-Only agent pursues a myopic, greedy strategy. In contrast, the FairMORL agent learns a more holistic and pedagogically sound strategy, balancing the need for mastery with the need to maintain engagement and build a solid foundation by recommending a varied and appropriately challenging set of skills. 5.5 Contribution 4: Auditing Bias with Skill Exposure Heatmaps To visualize content-level bias, we generated skill exposure heatmaps. These matrices show the relative frequency with which skills (grouped by difficulty and topic) were recommended to students in each of the three performance groups. The heatmap for the Mastery-Only RL policy revealed clear disparities. It showed a strong diagonal pattern, where low-difficulty skills were almost exclusively recommended to the low-performing group, and high-difficulty skills were almost exclusively recommended to the high-performing group. Crucially, it revealed that certain foundational algebra skills were being systematically under-recommended to the low-performing group, who needed them most, while advanced calculus skills were being over-recommended to the high-performing group, potentially leading to boredom. In contrast, the heatmap for the FairMORL policy showed a much more balanced distribution. While still personalized, the policy exposed all student groups to a wider range of skill difficulties, promoting both remediation for high-performers and opportunities for enrichment for low-performers. This simple visualization serves as a powerful diagnostic tool for educators and curriculum designers to audit the algorithmic curriculum for hidden biases. 5.6 Contribution 5: Mapping the Pareto-Optimal Frontier of Utility Fairness Trade-offs The "Balanced" FairMORL agent uses a single, fixed weight vector. To provide a more complete solution, we trained a set of FairMORL agents by systematically varying the weights assigned to the mastery () and fairness () objectives, while keeping the engagement weight constant. We then plotted the performance of each resulting policy in a 2D space of Mastery (AUC) vs. Fairness (SPD). The resulting plot illustrates the empirical Pareto front for this problem. It is not a straight line, confirming the limitations of linear scalarization and demonstrating a non-trivial, concave relationship between the objectives. This curve provides a "menu" of optimal policies for system designers and institutional stakeholders. For example, they can use the plot to answer concrete policy questions, such as: "How much mastery must we sacrifice to achieve a 20% reduction in fairness violations?" By visualizing the trade-offs, the Pareto front transforms the abstract task of setting weights into a concrete, value-driven decision about institutional priorities. It empowers stakeholders to select a policy that explicitly aligns with their educational philosophy, whether it is to maximize performance at all costs, prioritize equity above all, or choose a balanced point in between. 10 6. Discussion: Implications for Research and Practice 6.1 Challenging the Fairness-Utility Trade-off in Education The most significant implication of our findings is the empirical challenge to the widely assumed trade-off between fairness and utility. The observation that our fairness-aware agent, FairMORL, simultaneously improved both fairness and mastery suggests that, in complex socio-technical systems like education, these two goals are not necessarily in opposition. The relationship is more nuanced and can even be synergistic. We theorize that this synergy is mediated by psychological and pedagogical factors that are often abstracted away in purely technical models of fairness. For students, particularly those who are struggling or come from marginalized backgrounds, the perception of fairness is deeply intertwined with motivation, self-efficacy, and trust in the educational system. An ITS that is perceived as unfair—for example, by repeatedly recommending problems that are either frustratingly difficult or boringly simple—can lead to disengagement and learned helplessness. Conversely, a system that demonstrates fairness by providing varied, appropriately challenging, and supportive pathways may foster greater student trust and engagement. This increased engagement creates more opportunities for learning, leading to improved mastery. In this model, fairness is not a tax on performance but an investment in the underlying human conditions that enable learning. This finding calls for a shift in the research paradigm, from viewing fairness as a constraint to be managed to exploring it as a mechanism for enhancing pedagogical effectiveness. 6.2 Actionable Insights for Educators and ITS Designers Beyond its theoretical implications, the FairMORL framework and its associated analytical toolkit offer several actionable insights for practitioners in the field of educational technology. Each of our five contributions can be translated into a practical tool or process for building and maintaining more equitable ITS: ● Counterfactual Analysis as a "Cost of Bias" Metric: The counterfactual fairness analysis provides a tangible, interpretable metric for the impact of algorithmic bias. Instead of reporting an abstract statistical disparity like SPD, an institution can state, "Our old system resulted in an 18% loss in learning opportunity for struggling students." This framing, akin to a return-on-investment calculation, can be a powerful tool for administrators to justify investments in developing and deploying fairer technologies. ● Temporal Drift Monitoring for Continuous Assurance: The discovery of temporal fairness drift highlights that fairness is not a one-time check at deployment but an ongoing process of monitoring and maintenance. This implies that deployed educational technologies should be equipped with "fairness dashboards" that track metrics like EOP over time, alerting developers and educators to emerging biases just as they would monitor for system uptime or model accuracy degradation. ● Interpretability for Building Stakeholder Trust: The application of Integrated Gradients provides a mechanism for transparency. When a student, teacher, or parent questions a particular recommendation, this tool can generate a human-readable explanation (e.g., "The system recommended this easier skill because you have been working on the same topic for a while and took a long break, suggesting you might need a change of pace."). This ability to explain the "why" behind an AI's decision is crucial for fostering trust and enabling meaningful human-in-the-loop oversight. 31 ● Skill Heatmaps for Curriculum Auditing: The skill exposure heatmaps serve as a simple, powerful auditing tool for non-technical stakeholders. Curriculum designers and department heads can use these visualizations to quickly assess whether the algorithmic curriculum is providing equitable access to all topics across different student populations, allowing them to identify and rectify content-level biases. ● Pareto Fronts for Value-Driven Policy Making: The Pareto-optimal frontier empowers institutions to engage in explicit, value-driven conversations about their pedagogical priorities. Instead of a technical decision about reward weights, the choice becomes a policy discussion: "Are we, as an institution, willing to accept a 2% decrease in average mastery scores to ensure that our lowest-performing students receive 30% more equitable recommendations?" This makes the trade-offs explicit and places the decision in the hands of educational leaders rather than solely with AI developers. 6.3 Limitations and Avenues for Future Research While this work presents a significant step toward equitable adaptive tutoring, it is essential to acknowledge its limitations, which in turn point to promising directions for future research. ● Definition of Fairness Groups: Our study defines student groups based on their initial performance within the platform. While this is a relevant axis for pedagogical fairness, it is an endogenous variable—it is influenced by the system itself. Furthermore, it is not a legally protected attribute. Future research is critically needed to extend this framework to address fairness with respect to exogenous and protected demographic attributes such as race, gender, and socioeconomic status. This will require datasets with sensitive demographic information and careful consideration of privacy. ● Proximal Nature of the Engagement Reward: The engagement reward function is a composite of behavioral proxies (session length, dwell time, entropy). While these are grounded in educational data mining literature, they are not direct measures of a student's internal cognitive or affective state. A promising future direction would be to incorporate more direct measures, such as student self-reports or affective computing techniques. Alternatively, preference-based RL could be used to learn a reward function directly from student feedback, allowing the system to infer what constitutes an "engaging" experience for different individuals. 13 ● Offline Evaluation: This study was conducted using offline RL methods on a historical dataset. While this is a necessary and standard first step for validating the approach, it cannot fully capture the dynamic feedback loops that would occur in a live deployment. The definitive test of the FairMORL framework would be a large-scale, randomized controlled trial (A/B test) where students interact with different policies (e.g., Mastery-Only vs. FairMORL) in a real-world learning environment. ● Computational Complexity: Training MORL agents, especially a suite of them to map a Pareto front, is computationally intensive. Future work could explore more sample-efficient MORL algorithms or techniques for approximating the full Pareto front from a single, generalized policy network to make these methods more practical for widespread adoption. Conclusion This paper addressed the critical challenge of designing intelligent tutoring systems that are not only effective in promoting student mastery but are also engaging and equitable. We framed this challenge as a trilemma and proposed FairMORL, a multi-objective reinforcement learning framework that treats mastery, engagement, and fairness as co-equal, first-class objectives. By integrating fairness directly into the agent's reward signal, our approach moves beyond reactive, post-hoc audits and toward a proactive, fairness-by-design paradigm. Our empirical evaluation on the large-scale EdNet-KT1 dataset yielded a significant finding: the FairMORL agent not only drastically reduced fairness violations but also achieved superior performance in both student mastery and engagement compared to strong baselines. This result compellingly challenges the conventional notion of a direct fairness-utility trade-off in education, suggesting that a commitment to equity can be a powerful driver of pedagogical effectiveness. The five novel contributions presented—counterfactual fairness analysis, temporal drift monitoring, policy interpretability, skill exposure auditing, and Pareto-optimal trade-off analysis—constitute an integrated and comprehensive toolkit for the development, analysis, and governance of equitable AI in education. By providing our methodology and code as an open source, reproducible pipeline, we aim to facilitate further research and empower the educational technology community to build the next generation of ITS. Ultimately, this work demonstrates that by unifying the goals of effectiveness, engagement, and equity, we can create personalized learning systems that not only help individual students succeed but also contribute to a more just and inclusive educational landscape for all. References Liu, C., Xu, X., & Hu, D. (2015). Multiobjective reinforcement learning: a comprehensive overview . IEEE Transactions on Systems, Man, and Cybernetics: Systems, 45 (3), 385-398. (Elsevier) Abdolmaleki, A., Huang, S., Khadiv, M., Li, Y., & Righetti, R. (2021). A Survey on Multi Objective Deep Reinforcement Learning . In Reinforcement Learning for Big Data (pp. 1-28). (Springer) Yang, R., Sun, Y., & Todorov, E. (2019). A Unified Bellman Equation for Multi Objective Reinforcement Learning . Proceedings of the 36th International Conference on Machine Learning, PMLR 97 , 7103-7112. Pessach, D., & Shmueli, E. (2022). A review of fairness in machine learning . ACM Computing Surveys, 55 (3), 1-44. Baker, R. S., & Hawn, A. (2022). Algorithmic bias in education . International Journal of Artificial Intelligence in Education, 32 (4), 1052-1092. (Springer) Žliobaitė, I. (2017). Measuring discrimination in algorithmic decision making . Data Mining and Knowledge Discovery, 31 (4), 1060-1089. (Springer) Martin, J. G., & Arroyo, I. (2017). Agent-based pedagogical strategy adaptation in an exploratory learning environment . Computers & Education, 105 , 111-125. (Elsevier) Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014). A survey on concept drift adaptation . ACM Computing Surveys (CSUR), 46 (4), 1-37. Kordjamshidi, P., Bahrani, S., & van der Schaar, M. (2022). Fairness in sequential decision making under uncertainty . Expert Systems with Applications, 187 , 115933. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7793275","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":526621932,"identity":"860dd0b4-e976-465b-a485-ce1a01c888f4","order_by":0,"name":"REBBAH SIHAM","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA70lEQVRIiWNgGAWjYFACHjYGxgYGID6Q+ICBgZkELcyNBx4bkKaFvfngMwmitJi39x578HMHgz1v2+G0ap4aawb+9gOsG37g0SJz5ly6Ye8ZhsSZPcfSbvMcS2eQOJPAdrMHjxYJiRwzCd42hgTDGWfSbvM2HGZguMHAdoMHnxb5N2aSf9sY7O3vv/9WDNIiD9Ry8w9eW3jMpIG2MDY2HEhjBmkxAGq5jdcWnrx0Y9k2iUSglmTJOcfSeQzPJLbdlsGnhf3ssYdv22zsQVH54U2NtZzc8cPHbr7BowWmE87iYQBH0ygYBaNgFIwCigAACzlO4SSfOf8AAAAASUVORK5CYII=","orcid":"","institution":"","correspondingAuthor":true,"prefix":"","firstName":"REBBAH","middleName":"","lastName":"SIHAM","suffix":""}],"badges":[],"createdAt":"2025-10-06 17:08:30","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7793275/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7793275/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":93123285,"identity":"547ee5a9-c179-4577-b5ac-0ea0714c5639","added_by":"auto","created_at":"2025-10-09 10:03:54","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":407745,"visible":true,"origin":"","legend":"","description":"","filename":"again.docx","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/2731df9dabcd07d554ac585f.docx"},{"id":93122046,"identity":"7154771c-87ec-4945-8bd9-8ec5cf262593","added_by":"auto","created_at":"2025-10-09 09:47:54","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":4265,"visible":true,"origin":"","legend":"","description":"","filename":"41bcd08a4c4d470a8afa00d03c50bede.json","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/3b8928ee194db88ac27a21fc.json"},{"id":93122922,"identity":"8c746651-1eb9-40ce-a593-b1e6fad46693","added_by":"auto","created_at":"2025-10-09 09:55:54","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":95632,"visible":true,"origin":"","legend":"","description":"","filename":"41bcd08a4c4d470a8afa00d03c50bede1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/02891f14ab3497c0240082b8.xml"},{"id":93122927,"identity":"68b9c5ed-f729-4f88-b371-6792dc6451d7","added_by":"auto","created_at":"2025-10-09 09:55:54","extension":"png","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":73936,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/721e0691c9418d9fd2b7bc98.png"},{"id":93122924,"identity":"19ec71fa-d648-4906-ada2-43675c3c753c","added_by":"auto","created_at":"2025-10-09 09:55:54","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":126053,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/690af20b9cfc40a0f13ccb3c.png"},{"id":93122042,"identity":"12f5706a-5386-435d-991c-cefa6575bfd4","added_by":"auto","created_at":"2025-10-09 09:47:54","extension":"png","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":166096,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/0bb736345a8d7dd47e84d960.png"},{"id":93122034,"identity":"1fef497d-268a-4f67-8686-23ede863cdf0","added_by":"auto","created_at":"2025-10-09 09:47:54","extension":"png","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":64703,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/84b1d42bb1319b26f2edb8cd.png"},{"id":93123286,"identity":"6c6f10d1-3683-42e2-a545-0f38520ae907","added_by":"auto","created_at":"2025-10-09 10:03:54","extension":"png","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":34100,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/b12fca4ee44af3540ed01f2c.png"},{"id":93122038,"identity":"77f70611-655e-43ff-8530-c3a395cfcdf9","added_by":"auto","created_at":"2025-10-09 09:47:54","extension":"png","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":36323,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/b3a234708b07e76353209d3f.png"},{"id":93122925,"identity":"fe3c4082-80bd-46bf-84b2-2114c6e4432b","added_by":"auto","created_at":"2025-10-09 09:55:54","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":39603,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/b3ff0592a78802600692e204.png"},{"id":93122045,"identity":"ea6228b7-0050-4795-ba09-78eb4becbe96","added_by":"auto","created_at":"2025-10-09 09:47:54","extension":"png","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":22742,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/c24d67e42c322643622ac553.png"},{"id":93122041,"identity":"523ca0dc-55b1-4082-9726-d15be679a3e8","added_by":"auto","created_at":"2025-10-09 09:47:54","extension":"xml","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":90272,"visible":true,"origin":"","legend":"","description":"","filename":"41bcd08a4c4d470a8afa00d03c50bede1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/fafe75d008b0b53b23717326.xml"},{"id":93122044,"identity":"27516efe-f75b-4c5a-b9dc-411edce7027c","added_by":"auto","created_at":"2025-10-09 09:47:54","extension":"html","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":111224,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/b68723f8f9de2c628ca3bd87.html"},{"id":93122031,"identity":"a1157295-e492-4dc8-83f2-052b68a2f139","added_by":"auto","created_at":"2025-10-09 09:47:53","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":73936,"visible":true,"origin":"","legend":"\u003cp\u003eUnnumbered image in the Empirical Results and In-Depth Analysis section.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/683525eac7904e1d6d86ecae.png"},{"id":93122033,"identity":"acd531a9-24e4-4c7d-bc00-bc72c707c81c","added_by":"auto","created_at":"2025-10-09 09:47:53","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":148108,"visible":true,"origin":"","legend":"\u003cp\u003eUnnumbered image in the Empirical Results and In-Depth Analysis \u0026nbsp;section.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/f43ad3e488ae0549b6e17e20.png"},{"id":93122921,"identity":"d1b05d8c-ff36-427e-8d70-b56a05b867e5","added_by":"auto","created_at":"2025-10-09 09:55:53","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":132973,"visible":true,"origin":"","legend":"\u003cp\u003eUnnumbered image in the Empirical Results and In-Depth Analysis section.\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/e03d06b7166c5c057439a112.png"},{"id":93122048,"identity":"e64f95a6-8037-4102-b7f6-d8640aa49afd","added_by":"auto","created_at":"2025-10-09 09:47:54","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":64703,"visible":true,"origin":"","legend":"\u003cp\u003eUnnumbered image in the Discussion: Implications for Research and Practice section.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/348b96ab00437cb2157f9093.png"},{"id":98775049,"identity":"9f19669a-b009-44e5-8bd0-5d81d98da1fb","added_by":"auto","created_at":"2025-12-22 12:18:07","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2555649,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7793275/v1/118f8910-7dbf-457b-adcc-c560fc92b392.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"FairMORL: A Framework for Synergistic Optimization of Mastery, Engagement, and Equity in Adaptive Tutoring","fulltext":[{"header":"1. Introduction: The Trilemma of Personalization in AI Powered Tutoring ","content":"\u003cp\u003e\u003cstrong\u003e1.1 The Promise and Peril of RL-based Intelligent Tutoring Systems \u0026nbsp; (ITS)\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe integration of Artificial Intelligence (AI) into education has heralded a new era of \u0026nbsp;personalized learning, moving beyond the one-size-fits-all model of traditional instruction.\u003csup\u003e1\u003c/sup\u003e From \u0026nbsp;the early days of computer-assisted instruction in the 1960s to the sophisticated adaptive \u0026nbsp;learning platforms of today, the goal has been to tailor educational experiences to the unique \u0026nbsp;pace, style, and knowledge level of each student.\u003csup\u003e1\u003c/sup\u003e At the forefront of this technological evolution \u0026nbsp;is Reinforcement Learning (RL), a paradigm of machine learning adept at optimizing sequential \u0026nbsp; decision-making under uncertainty.\u003csup\u003e2\u003c/sup\u003e In the context of Intelligent Tutoring Systems (ITS), RL \u0026nbsp;agents can learn complex policies to dynamically select the next best learning item, problem, or \u0026nbsp;pedagogical intervention for a student, promising a level of individualized instruction previously \u0026nbsp;achievable only through one-on-one human tutoring.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eHowever, this promise is shadowed by a significant peril: the pervasive issue of algorithmic bias. \u0026nbsp;As AI systems are increasingly deployed in high-stakes domains such as hiring, criminal justice, \u0026nbsp;and finance, a growing body of evidence reveals their capacity to inherit, perpetuate, and even \u0026nbsp;amplify societal biases present in historical data.\u003csup\u003e4\u003c/sup\u003e The educational domain is not immune to this \u0026nbsp;risk. An RL-based tutor trained on data from a diverse student population may inadvertently \u0026nbsp; learn policies that favor students from majority groups or those who are already high-achieving, \u0026nbsp;while marginalizing struggling learners or those from underrepresented backgrounds.\u003csup\u003e1\u003c/sup\u003e Such \u0026nbsp;biases can create feedback loops where disadvantaged students are offered less challenging \u0026nbsp;material, fall further behind, and have their initial disadvantage reinforced by the very system \u0026nbsp;designed to help them. The gravity of this issue demands a proactive approach to fairness in the \u0026nbsp;design of educational AI.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e1.2 Beyond Single-Objective Myopia: The Need to Unify Mastery, \u0026nbsp;Engagement, and Equity\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA fundamental limitation of many current RL-driven ITS is their \u0026quot;single-objective myopia\u0026quot;\u0026mdash;a \u0026nbsp;design philosophy that defines success almost exclusively through the lens of student mastery.\u003csup\u003e3\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eThe agent\u0026apos;s reward function is often engineered to maximize the probability of a student \u0026nbsp;answering the next question correctly, a proxy for knowledge acquisition. While crucial, this \u0026nbsp;narrow focus overlooks other essential dimensions of a successful learning experience, creating \u0026nbsp;a pedagogical imbalance. This paper argues that a truly effective ITS must navigate a complex \u0026nbsp;trilemma by simultaneously optimizing for three distinct yet interconnected pillars:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e1.\u0026nbsp;\u003cstrong\u003eMastery:\u0026nbsp;\u003c/strong\u003eThe traditional objective, representing the student\u0026apos;s developing knowledge and \u0026nbsp;skill proficiency. This remains the foundational goal of any educational system.\u0026nbsp;2.\u0026nbsp;\u003cstrong\u003eEngagement:\u0026nbsp;\u003c/strong\u003eA critical predictor of long-term learning, motivation, and persistence. A \u0026nbsp;system that maximizes short-term correctness at the cost of making learning monotonous \u0026nbsp; or frustrating is ultimately a pedagogical failure. Engagement encompasses factors like \u0026nbsp;student interest, effort, and sustained interaction with the learning platform.\u0026nbsp;3.\u0026nbsp;\u003cstrong\u003eEquity:\u0026nbsp;\u003c/strong\u003eThe principle of ensuring that the system\u0026apos;s pedagogical benefits are distributed \u0026nbsp;fairly across all student subgroups, irrespective of their initial performance level, \u0026nbsp;demographic background, or other sensitive attributes. An equitable system actively works \u0026nbsp; to close, rather than widen, pre-existing achievement gaps.\u003csup\u003e1\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eThe challenge lies in the inherent tensions between these objectives. An aggressive focus on \u0026nbsp;mastery might lead to recommending overly difficult problems that disengage a struggling \u0026nbsp; student. A policy designed purely for engagement might recommend entertaining but \u0026nbsp;pedagogically shallow content. A naive attempt to enforce equity might compromise \u0026nbsp;personalization for all. Resolving this trilemma requires a more sophisticated framework that can \u0026nbsp;explicitly model and balance these competing goals.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e1.3 Limitations of Existing Fairness-Aware RL: From Post-Hoc Audits \u0026nbsp;to Integrated Objectives\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe field of AI in education has begun to grapple with the challenge of fairness. Initial efforts \u0026nbsp;largely focused on post-hoc audits\u0026mdash;analyzing a trained model\u0026apos;s predictions to detect statistical \u0026nbsp;biases after the fact.\u003csup\u003e6\u003c/sup\u003e While useful for diagnosis, this approach is reactive rather than proactive. \u0026nbsp;More recent and advanced methods have sought to integrate fairness directly into the model \u0026nbsp;training process, often by formulating it as a constraint on the optimization problem or as an \u0026nbsp;auxiliary penalty term in the loss function.\u003csup\u003e7\u003c/sup\u003e These \u0026quot;fairness-aware\u0026quot; methods represent a \u0026nbsp;significant step forward, forcing the learning algorithm to consider equity during policy \u0026nbsp;development.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eHowever, treating fairness as a secondary constraint or penalty can be limiting. It frames equity \u0026nbsp;as a boundary condition to be satisfied while optimizing a primary utility function (e.g., mastery), \u0026nbsp;rather than as a desirable outcome in its own right. This paper posits a different approach, \u0026nbsp; arguing that in the complex socio-technical context of education, fairness should be elevated to \u0026nbsp;a \u003cstrong\u003efirst-class objective, co-equal with mastery and engagement\u003c/strong\u003e. By embedding fairness\u0026nbsp;\u003c/p\u003e\n\u003cp\u003edirectly into the agent\u0026apos;s reward signal within a Multi-Objective Reinforcement Learning (MORL) \u0026nbsp;framework, we can train policies that learn to actively seek out and promote equitable \u0026nbsp; outcomes, rather than merely avoiding egregious inequity. This \u0026quot;fairness-by-design\u0026quot; philosophy \u0026nbsp; represents a conceptual shift toward building inherently equitable systems from the ground up.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e1.4 Our Contributions: An Integrated Solution to the Tutoring \u0026nbsp; Trilemma\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis paper introduces \u003cstrong\u003eFairMORL\u003c/strong\u003e, a novel framework designed to resolve the tutoring trilemma \u0026nbsp;by synergistically optimizing mastery, engagement, and equity. The core of our work is an \u0026nbsp;integrated set of five contributions, which collectively provide a comprehensive toolkit for \u0026nbsp;building, analyzing, and deploying fair and effective adaptive tutoring systems.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e1.\u0026nbsp;\u003cstrong\u003eCounterfactual Fairness Analysis:\u0026nbsp;\u003c/strong\u003eWe move beyond correlational fairness metrics to \u0026nbsp;quantify the \u003cem\u003ecausal\u0026nbsp;\u003c/em\u003eimpact of a student\u0026apos;s performance group on the learning opportunities \u0026nbsp;they receive. This analysis reveals the extent to which a student\u0026apos;s outcome would have \u0026nbsp;changed had the system treated them differently, providing a powerful measure of \u0026nbsp; algorithmic bias.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e2.\u0026nbsp;\u003cstrong\u003eTemporal Fairness Drift Monitoring:\u0026nbsp;\u003c/strong\u003eRecognizing that ITS are dynamic, long-lived \u0026nbsp;systems, we introduce a methodology for monitoring fairness metrics over time. This allows \u0026nbsp;for the detection of \u0026quot;fairness drift,\u0026quot; where a policy that was fair at deployment becomes \u0026nbsp;inequitable as student populations or behaviors change.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e3.\u0026nbsp;\u003cstrong\u003ePolicy Interpretability via Integrated Gradients:\u0026nbsp;\u003c/strong\u003eTo address the \u0026quot;black box\u0026quot; problem and \u0026nbsp;build stakeholder trust, we apply an axiomatically-grounded interpretability technique to the \u0026nbsp;RL agent\u0026apos;s policy network. This reveals \u003cem\u003ewhy\u0026nbsp;\u003c/em\u003ethe agent makes specific recommendations for \u0026nbsp;individual students, making its logic transparent to educators and developers.\u003csup\u003e6\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003e4.\u0026nbsp;\u003cstrong\u003eSkill Exposure Auditing:\u0026nbsp;\u003c/strong\u003eWe propose skill exposure heatmaps as a practical diagnostic \u0026nbsp;tool. These visualizations provide an at-a-glance summary of which skills are being over- or \u0026nbsp;under-recommended to different student groups, enabling rapid identification of content level biases.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e5.\u0026nbsp;\u003cstrong\u003ePareto-Optimal Trade-off Analysis:\u0026nbsp;\u003c/strong\u003eInstead of committing to a single, fixed balance \u0026nbsp;between objectives, we map out the Pareto-optimal frontier. This provides system \u0026nbsp; designers and educational stakeholders with a \u0026quot;menu\u0026quot; of optimal policies, allowing them to \u0026nbsp;make informed, value-driven decisions about the precise trade-offs between mastery, \u0026nbsp;engagement, and equity.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTogether, these contributions form a robust, end-to-end methodology for developing the next \u0026nbsp;generation of ITS\u0026mdash;systems that are not only personalized but also principled, transparent, and \u0026nbsp;equitable by design.\u003c/p\u003e"},{"header":"2. Theoretical Foundations and Related Work","content":"\u003cp\u003eTo situate our contributions, this section provides a structured review of the theoretical \u0026nbsp;foundations and related literature, creating a funnel of increasing specificity. We begin with the \u0026nbsp;broad fields of Multi-Objective Reinforcement Learning (MORL) and fairness in educational AI, \u0026nbsp; then narrow our focus to the cutting-edge concepts of counterfactual fairness, temporal fairness \u0026nbsp;drift, and RL interpretability, which form the direct underpinnings of our novel contributions.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.1 Multi-Objective Reinforcement Learning (MORL): From Linear \u0026nbsp; Scalarization to Pareto Optimization\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eStandard RL problems are formalized as Markov Decision Processes (MDPs), where an agent \u0026nbsp;learns a policy to maximize a scalar reward signal.\u003csup\u003e3\u003c/sup\u003e However, many real-world problems, \u0026nbsp;including adaptive tutoring, involve multiple, often conflicting, objectives. MORL extends the \u0026nbsp;MDP framework to a Multi-Objective MDP (MOMDP), formally defined by the tuple.\u003csup\u003e3\u003c/sup\u003e Here, the reward function \u0026nbsp;is vector-valued, returning a separate reward for each of the objectives. Consequently, the \u0026nbsp; value of a policy , , is also a vector representing the expected discounted cumulative reward for \u0026nbsp;each objective.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe most common and straightforward method for solving a MOMDP is \u003cstrong\u003elinear scalarization\u003c/strong\u003e, where the reward vector is converted into a scalar reward through a weighted sum: , where is a \u0026nbsp;vector of predefined weights. The problem is then reduced to a standard single-objective RL \u0026nbsp; problem. While simple, this approach has a significant theoretical limitation: it can only find \u0026nbsp;policies that lie on the \u003cem\u003econvex hull\u0026nbsp;\u003c/em\u003eof the Pareto front.\u003csup\u003e10\u003c/sup\u003e It is incapable of discovering optimal \u0026nbsp;policies that exist in concave regions of the objective space, meaning a potentially large set of \u0026nbsp;desirable trade-offs may be inaccessible.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTo overcome this, modern MORL research focuses on approximating the entire \u003cstrong\u003ePareto front\u003c/strong\u003e: \u0026nbsp;the set of all non-dominated solutions where no single objective can be improved without \u0026nbsp;degrading at least one other objective.\u003csup\u003e3\u003c/sup\u003e Approximating the Pareto front provides decision makers with a complete set of optimal trade-off policies, from which they can select the one that \u0026nbsp;best aligns with their preferences.\u003csup\u003e9\u003c/sup\u003e An emerging subfield, preference-based MORL (Pb-MORL), \u0026nbsp;further abstracts this process by learning a reward model directly from human preferences (e.g., \u0026nbsp;pairwise comparisons of trajectories), thereby avoiding the difficult task of manually engineering \u0026nbsp;a multi-objective reward function.\u003csup\u003e13\u003c/sup\u003e While our work uses an explicitly defined reward function, \u0026nbsp;the broader goal of exploring the Pareto front aligns with the state-of-the-art in MORL.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.2 Defining and Measuring Fairness in Educational AI: A Review of \u0026nbsp;Group and Individual Notions\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study of fairness in AI has produced a rich taxonomy of definitions and metrics, which can \u0026nbsp;be broadly categorized into group and individual fairness. These concepts arise from the need \u0026nbsp;to mitigate various sources of bias, which can be data-related (e.g., underrepresentation of \u0026nbsp;certain groups), algorithmic (e.g., objective functions that inadvertently favor the majority), or \u0026nbsp;related to user interaction.\u003csup\u003e1\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eGroup fairness\u0026nbsp;\u003c/strong\u003enotions require that a model\u0026apos;s behavior is statistically equivalent across \u0026nbsp;different demographic groups. The most common metrics, which we employ in this work, \u0026nbsp; include:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eStatistical Parity (or Demographic Parity):\u0026nbsp;\u003c/strong\u003eThis metric requires that the probability of \u0026nbsp;receiving a particular outcome (e.g., being recommended a specific skill) is independent of \u0026nbsp;group membership. A model satisfies statistical parity if for two groups and . The \u0026nbsp; Statistical Parity Difference (SPD) measures the violation of this condition.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eEqual Opportunity:\u0026nbsp;\u003c/strong\u003eThis metric is less stringent and requires that the true positive rates \u0026nbsp;are equal across groups. In an educational context, this could mean that among all \u0026nbsp;students who are capable of mastering a skill, the probability of being recommended that \u0026nbsp;skill is the same regardless of their group. A model satisfies equal opportunity if . The Equal \u0026nbsp;Opportunity Difference (EOP) measures the violation.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWhile group fairness metrics are essential for auditing system-level biases, they can obscure \u0026nbsp;injustices at the individual level. \u003cstrong\u003eIndividual fairness\u0026nbsp;\u003c/strong\u003enotions, in contrast, demand that similar \u0026nbsp; individuals are treated similarly. This seemingly simple principle is difficult to formalize, as it \u0026nbsp;requires a meaningful metric of \u0026quot;similarity.\u0026quot; This challenge has led to the development of more \u0026nbsp;rigorous, causality-based approaches.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eSurveys of experts in AI for Education have underscored the need for fairness frameworks that \u0026nbsp;are context-specific, integrated throughout the entire AI development pipeline, and centered on \u0026nbsp;human evaluation, rather than relying solely on statistical metrics.\u003csup\u003e6\u003c/sup\u003e The FairMORL framework, \u0026nbsp;with its suite of diagnostic and analytical tools, is designed in direct response to these \u0026nbsp;community-identified needs.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.3 Causal Perspectives on Fairness: The Emergence of \u0026nbsp; Counterfactual Approaches\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eCounterfactual fairness has emerged as a powerful and rigorous definition of individual fairness \u0026nbsp;grounded in the theory of causal inference.\u003csup\u003e15\u003c/sup\u003e It moves beyond statistical correlations to ask \u0026nbsp;causal questions about fairness. The concept is formally defined using Structural Causal Models \u0026nbsp;(SCMs), which represent causal relationships between variables.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDefinition (Counterfactual Fairness):\u0026nbsp;\u003c/strong\u003eA predictor is counterfactually fair if, for any individual, \u0026nbsp;its prediction would not have changed had their sensitive attribute been different, while holding \u0026nbsp;all non-descendant variables constant.\u003csup\u003e15\u003c/sup\u003e This is formally expressed as:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003ewhere the subscript denotes a causal intervention (using Pearl\u0026apos;s do()-operator) that sets the \u0026nbsp;sensitive attribute to a counterfactual value .\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis definition provides an intuitive and powerful guarantee: the model\u0026apos;s decision is not based \u0026nbsp;on an individual\u0026apos;s group membership in any causal sense. Recent research has begun to \u0026nbsp; explore how to apply this concept to sequential decision-making settings and reinforcement \u0026nbsp;learning, with much of the pioneering work focusing on high-stakes domains like healthcare, \u0026nbsp;where it is used to ensure equitable allocation of treatments over time.\u003csup\u003e18\u003c/sup\u003e However, its application \u0026nbsp;to the domain of RL-based adaptive tutoring remains largely unexplored. To our knowledge, this \u0026nbsp;paper is the first to employ counterfactual analysis to quantify opportunity loss in an adaptive \u0026nbsp;tutoring policy, representing a significant novel contribution to the field.\u003csup\u003e15\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.4 The Dynamics of Bias: Characterizing Temporal Fairness Drift in \u0026nbsp; Deployed Systems\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eMachine learning models are often trained and evaluated on static datasets, under the \u0026nbsp;assumption that the data distribution will remain stable upon deployment. In reality, this \u0026nbsp; assumption is frequently violated. The phenomenon of \u003cstrong\u003econcept drift\u0026nbsp;\u003c/strong\u003edescribes the change in \u0026nbsp;the statistical properties of the input data or the relationship between input features and the \u0026nbsp;target variable over time, which can lead to significant degradation in model performance.\u003csup\u003e22\u003c/sup\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn the context of fairness, this temporal instability gives rise to \u003cstrong\u003efairness drift\u003c/strong\u003e: a model that was \u0026nbsp;deemed fair according to certain metrics at the time of training may become progressively more \u0026nbsp;biased after deployment.\u003csup\u003e24\u003c/sup\u003e This is a critical concern for real-world systems like an ITS, which \u0026nbsp;may operate continuously for months or years.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eA more precise and recently formalized concept is \u003cstrong\u003egroup-specific concept drift\u003c/strong\u003e. This occurs \u0026nbsp;when the underlying data distribution changes for one specific demographic group but remains \u0026nbsp;stable for others.\u003csup\u003e4\u003c/sup\u003e Formally, for a sensitive attribute\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e, this drift is present if for a specific group , the conditional probability distribution changes over \u0026nbsp;time, i.e., , while for all other groups , .\u003csup\u003e4\u003c/sup\u003e For example, the study habits and background\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eknowledge of students enrolling in an online course in the fall may differ systematically from \u0026nbsp;those enrolling in the spring. A static policy trained on fall data may exhibit fairness drift when \u0026nbsp;applied to the spring cohort. This under-explored phenomenon poses a substantial threat to \u0026nbsp;long-term equity in educational AI, and our second contribution provides a necessary \u0026nbsp; methodology for its detection and monitoring.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e2.5 Opening the Black Box: Interpretability in Deep Reinforcement \u0026nbsp; Learning\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe increasing complexity of deep learning models, including deep RL agents, has led to the \u0026nbsp;\u0026quot;black box\u0026quot; problem, where the internal logic of a model\u0026apos;s decisions is opaque to human users. \u0026nbsp;In high-stakes applications like education, this lack of transparency is a major barrier to \u0026nbsp; adoption, as it erodes trust, hinders debugging, and complicates accountability.\u003csup\u003e28\u003c/sup\u003e Research has \u0026nbsp;shown that students and educators are more likely to perceive AI systems as fair and \u0026nbsp; trustworthy if their decision-making processes are transparent and explainable.\u003csup\u003e30\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eFeature attribution methods are a prominent class of interpretability techniques that aim to \u0026nbsp;explain a model\u0026apos;s output by assigning an importance score to each of its input features. While \u0026nbsp;methods like SHAP (SHapley Additive exPlanations) are widely used\u0026nbsp;\u003csup\u003e32\u003c/sup\u003e, our work focuses on\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eIntegrated Gradients (IG)\u003c/strong\u003e. IG is built on a strong axiomatic foundation, including properties like \u0026nbsp;Sensitivity (if a feature is changed in a way that affects the output, its attribution should be non zero) and Completeness (the sum of attributions should equal the difference between the \u0026nbsp;model\u0026apos;s output for the input and a baseline).\u003csup\u003e33\u003c/sup\u003e Intuitively, IG calculates feature importance by \u0026nbsp;accumulating the gradients of the output with respect to the feature along a straight-line path \u0026nbsp;from a baseline input (e.g., a zero vector) to the actual input.\u003csup\u003e36\u003c/sup\u003e This path-integration method \u0026nbsp;provides a more robust attribution than simply using the raw gradient at the input point.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eDespite the theoretical appeal of IG, there is a lack of established methodologies or clear \u0026nbsp;tutorials for applying it to the distinct components of an actor-critic RL agent\u0026mdash;namely, the policy \u0026nbsp;network (the actor) and the value network (the critic).\u003csup\u003e32\u003c/sup\u003e Our third contribution addresses this gap \u0026nbsp;by providing a practical demonstration of how IG can be adapted to analyze the policy network, \u0026nbsp;offering interpretable insights into the agent\u0026apos;s decision-making logic in an educational context.\u0026nbsp;\u003c/p\u003e"},{"header":"3. The FairMORL Framework: A Multi-Objective Approach to Equitable Tutoring","content":"\u003cp\u003e\u003cstrong\u003e3.1 System Architecture and Problem Formulation as a Multi Objective MDP (MOMDP)\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe FairMORL framework is designed as a closed-loop system where a knowledge tracing \u0026nbsp;model and a multi-objective reinforcement learning agent work in concert to personalize the \u0026nbsp;learning experience. The system architecture consists of two primary components:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e1. \u003cstrong\u003eDeep Knowledge Tracing (DKT) Module:\u0026nbsp;\u003c/strong\u003eThis component serves as the state estimator. \u0026nbsp;It takes the sequence of a student\u0026apos;s past interactions (question-answer pairs) and produces \u0026nbsp;a latent representation of their current knowledge state. This state is then used by the RL \u0026nbsp;agent.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e2. \u003cstrong\u003eFair MORL Agent:\u0026nbsp;\u003c/strong\u003eThis is the decision-making component. It uses an actor-critic \u0026nbsp;architecture to learn a policy that recommends the next skill for the student to practice. Its \u0026nbsp;goal is to maximize a vector of rewards that balances mastery, engagement, and fairness.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWe formally model this sequential decision-making problem as a Multi-Objective Markov \u0026nbsp;Decision Process (MOMDP), defined by the tuple , where:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● : The state space, representing the agent\u0026apos;s perception of the student\u0026apos;s current learning \u0026nbsp;status.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● : The action space, comprising the set of all skills or knowledge components that can be \u0026nbsp; recommended.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● : The state transition probability function, , which is implicitly defined by the student\u0026apos;s \u0026nbsp; response to the recommended skill in state .\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● : A discount factor, $\\gamma \\in $, which balances the importance of immediate versus \u0026nbsp;future rewards.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● : A vector-valued reward function, , which provides feedback on the quality of the action \u0026nbsp; taken in state across the three core objectives.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe agent\u0026apos;s goal is to learn a policy that maximizes the expected discounted cumulative reward \u0026nbsp;vector.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3.2 State and Action Space Representation for Adaptive Tutoring\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe effectiveness of an RL agent is highly dependent on a well-designed state representation \u0026nbsp;that captures all relevant information for decision-making. The state vector in the FairMORL \u0026nbsp;framework is a concatenation of four features, each chosen for its pedagogical relevance:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● The components are defined as follows:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003eskill_idx\u003c/strong\u003e: The categorical index of the skill or knowledge component associated with the \u0026nbsp;student\u0026apos;s most recent interaction. This provides immediate context about what the student \u0026nbsp;has just worked on.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003emastery_pred\u003c/strong\u003e: A vector representing the DKT model\u0026apos;s predicted probability of the student \u0026nbsp;correctly answering a question from each available skill in the next time step. This feature \u0026nbsp;is the primary input for the mastery objective. We utilize the foundational Deep Knowledge \u0026nbsp;Tracing model with LSTMs [Piech et al., 2015] for its proven ability to model latent \u0026nbsp; knowledge states, while acknowledging that more recent attention-based architectures like \u0026nbsp;AKT [Ghosh et al., 2020] or SAINT \u003csup\u003e39\u003c/sup\u003e could be substituted in future work.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003esession_entropy\u003c/strong\u003e: The Shannon entropy calculated over the distribution of skills practiced \u0026nbsp;by the student within the current learning session. A higher entropy indicates greater \u0026nbsp;variety in the practiced skills. This feature is included as a proxy for exploration and to \u0026nbsp; prevent monotonous \u0026quot;drilling\u0026quot; on a single topic, which can negatively impact student \u0026nbsp;engagement.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003etime_delta_norm\u003c/strong\u003e: The normalized time elapsed since the student\u0026apos;s last interaction. The \u0026nbsp;time between interactions is a known predictor in knowledge tracing models, potentially \u0026nbsp;indicating forgetting or disengagement.\u003csup\u003e40\u003c/sup\u003e Normalizing this feature ensures it is on a \u0026nbsp;comparable scale with other state components.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe \u003cstrong\u003eaction space\u0026nbsp;\u003c/strong\u003eis discrete, defined as the set of all unique skill indices available in the \u0026nbsp;curriculum, . An action corresponds to the agent\u0026apos;s decision to recommend a problem \u0026nbsp;associated with a specific skill.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3.3 Designing the Vector-Valued Reward Function\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe central innovation of the FairMORL framework lies in its explicit, multi-objective reward \u0026nbsp;function. Each component is carefully designed to provide a distinct learning signal to the agent, \u0026nbsp;guiding it toward policies that are effective, engaging, and equitable.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3.3.1 Quantifying Mastery ()\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe mastery reward incentivizes the agent to make recommendations that lead to positive \u0026nbsp;learning outcomes. It is defined based on the student\u0026apos;s actual performance and the DKT \u0026nbsp; model\u0026apos;s prediction. Let pt be the DKT\u0026apos;s predicted probability of the student correctly answering \u0026nbsp;the question for the chosen skill at, and let ct {0,1} be the actual correctness of the student\u0026apos;s \u0026nbsp;response. The mastery reward is then given by:\u003c/p\u003e\n\u003cp\u003eThis reward is maximized (approaches 1) when the DKT model\u0026apos;s prediction is accurate. This \u0026nbsp;encourages the agent to select items where the student\u0026apos;s learning trajectory is well-understood, \u0026nbsp;either by recommending items the student is ready to master (high pt, ct=1) or items that are \u0026nbsp;appropriately challenging and result in a learning opportunity from an incorrect answer (low pt, \u0026nbsp;ct=0).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3.3.2 Modeling Engagement ()\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe engagement reward is a composite metric designed to capture multiple facets of student \u0026nbsp;involvement. It is a weighted sum of three behavioral proxies:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003eentropy\u003c/strong\u003e: The session entropy, as described in the state representation. Rewarding higher \u0026nbsp;entropy encourages the policy to introduce variety and prevent pedagogical stagnation.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003edwell_time\u003c/strong\u003e: The normalized time the student spends on the recommended question. Very short times may indicate guessing or disengagement, while very long times may indicate \u0026nbsp;frustration. The reward is structured to be maximal for a moderate dwell time, incentivizing \u0026nbsp;the agent to recommend items of appropriate difficulty.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003esession_length\u003c/strong\u003e: The normalized length of the current learning session. A positive reward \u0026nbsp;for longer sessions directly encourages the agent to learn policies that keep students \u0026nbsp;engaged and learning for extended periods.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3.3.3 Embedding Statistical Parity as a First-Class Objective ()\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis component directly integrates a group fairness metric into the RL optimization loop, \u0026nbsp;aligning with modern approaches that encode fairness into the reward signal itself.\u003csup\u003e8\u003c/sup\u003e We define \u0026nbsp;the fairness reward based on the Statistical Parity Difference (SPD) in skill recommendations \u0026nbsp;between high-performing and low-performing student groups. Let\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eP(at G=g) be the probability of recommending skill at to a student from group g {high,low}. \u0026nbsp;The fairness reward is the negative absolute difference, calculated over a sliding window of \u0026nbsp; recent interactions to ensure responsiveness:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eBy rewarding the agent for actions that reduce this disparity, the policy is explicitly trained to \u0026nbsp;allocate learning opportunities more equitably across performance groups. This transforms \u0026nbsp;fairness from a passive constraint to an active optimization goal.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e3.4 Policy Optimization via Actor-Critic Methods\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo learn a policy that optimizes the scalarized combination of these three objectives, we employ \u0026nbsp;an Actor-Critic (AC) algorithm. AC methods are well-suited for this task as they combine the \u0026nbsp;benefits of policy-based methods (ability to handle stochastic policies and continuous action \u0026nbsp;spaces, though our action space is discrete) and value-based methods (reduced variance and \u0026nbsp; improved sample efficiency).\u003csup\u003e42\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eThe architecture consists of two neural networks:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003eThe Actor (Policy Network):\u0026nbsp;\u003c/strong\u003eThis network takes the current state as input and outputs a \u0026nbsp;probability distribution over the action space, , where are the network parameters. Actions \u0026nbsp;are sampled from this distribution.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003eThe Critic (Value Network):\u0026nbsp;\u003c/strong\u003eThis network also takes the state as input but outputs a \u0026nbsp;three-dimensional vector, , representing the estimated expected future return for each of \u0026nbsp;the three objectives from that state.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTraining proceeds by updating both networks. The critic is updated using the temporal \u0026nbsp;difference (TD) error for each objective. The actor is updated using the policy gradient theorem, \u0026nbsp;where the \u0026quot;advantage\u0026quot;\u0026mdash;how much better an action was than expected\u0026mdash;is provided by the \u0026nbsp;critic. The final objective for the gradient update is a linear scalarization of the individual \u0026nbsp;objective advantages, governed by a weight vector w=[w1,w2,w3]:\u0026nbsp;\u003c/p\u003e\u003cp\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAegAAAAeCAYAAADn7I28AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAxWSURBVHhe7ZxBix3HEcfH+RaStTo5n0AOwitwsMhVAYGTFTjERwcdHCwrEIOOAgfiSMQHER8dbLIiBkF0DXJkkEDE+gKRT1pn9TEU/2bfPyqVu2dqZt6uZp7rB8Pbea+np6q6uqu6ZqSXnn5HkyRJkiTJrPjR6jNJkiRJkhmRATpJkiRJZkgG6CRJkiSZIRmgkyRJkmSGZIBOkiRJkhmSATpJkiRJZkgG6CRJkiSZIaEA/eibb5qdCxdWZ2UibZIkSZJkEyH+EQe7iLSx9Abo7e0zzRtvnG3u3bvfnNg62dz96qvVL8/oanP58u+a3Zs3V2frhb65dxfXrl9vZToM0JO+Of55505RFgYDOcU65aHfIYONc3BvK08UdCtdhz5yurnrT1vu3XcN+pT8fNNAR3Q9Suz9NB5d64PmWNd4RNoI+Z98FR/ivHZIXj45HzN3BPekD82ZiLyesfMHG0sn7D73uWptJfzY1UBXP05DfMRi/YNrOaxMILmIf8TBko9E2pToDNAYYuvkVvPll3eaM2e2m48++mPz1lu/ek7Jvjav/uTV5ut/f93+vWl88PsPmgs7O823e4+bn509237nnfDtX7+9+mu94IRDE5+bu7vN559/1l43ZLLUaJ312vW23x+/8kr73Zz1R0b8s0+m06dPNw8fPlydbS4/ff31dsE4KlikjvJ+HnwTf8UH7t+/t/q2aU5ubbVzuHTg2zB17qD74729ts9L77032se4lj6GQkC4dOng2iXMVcaHcWK8hHTHjl39nTp16rvrp/sZ6xv3Ydy5L/OF2GZhXJFR8Y9PrrEBONKmRjVAIxyGuHjx4uqbpg1GOPPtf9xuzyNtOB86OEsB3Y+/fHx11jRv/uLN5vbtA73nyvHjB/Lu7++3n1O4ceNGO+nFEvTHH6HLJ8+dO9d88fcvVmebzSbPT498/tixY+3nUKbMnf3/7rfrojhKH1MglvywhLmqcfKJBHbsGgMSEDaN2iQeJowhc0hJD7Am2jkVaVOjt8RtBxWU2Vj62pA1WGORUbDzRkCVDzj8QHCN/b3UBshE9Htf+QNse44pZT6rO5nbgwcPVmcH1QVle9zHgg2sDF4vzu3vVkb6U/ZFuYS+hEpxOmpZ2pMnT1Z/jYOxYTfEQiOWoj+L08d//nh19n1qOwzvrxwW7s39rAxWboi04XcO7MfvVgd9p8PCufd/3cfrInyFS3POzz10998NsQ9jpN0Q39NWsNjaa+x4Cnabtg2yeCJtwK9XQxk7dwgaouRjEd/QvBmDTUyWtFaVgjEJTxeRCoX3Z3t/5NRumU/JCvixnWd2kwalKkekTYlqgJYTf/KXT9rPEpE2UDIWznDr1q1WSA6CuC2xYDwMo/ICB1kIg2yhn71vD0pHHEwCv0hZGATKH2rPQaCpOUeN0oLnS4YkKmR7yM19LEwO3d/rTt/oqZKUrpXj058SIMolDDbod12jUoqdFDaLmwIVEuS2/S1BfyCpwG9qCzhsb28/57NaaOhT/SO39zXabZ14VjLde7z3f7lEtM27v323/V26skDg37oO+/CdfBHZ0Ev60gfjwRyqjXutHMjc032wN7rzSEff+THrsw9jhLygPoXKf3yPrpz7sfFtkG9om6lJ6ZS5wxrl8T4GEd8YSinALWGu1hIpm+jU8AmIR/EF2SQD80BxADmZN8AnjzikG7piL0AW/K6LSJsa1QCNMyI8RmMAGEy/0EXaQM1YV69eXf3VtGVyFhctNpRPcQYcSZDtg5+Yek4E9Ek/yOShb75n4bNwn7HPLHy5jL68fCW6dCfhYbLImYH22LfWN9/zu33cwPjQT8n2Y8p0FuzFQuJZgv5aaLsybL+zZMfNYqNrgcTTyg3cTwsSfPiHD78nd6QNPBfEVvPK+jr2oS8lyMjG4sFigEwsNvRh55BH+lgdwMqneYec4vz5863uImqfEsis6+gDnfzYII9tg5+xRlgibcDKCMioXZQ/Sv42du74+VJ6PyfqG2PwAW8pa1UpsSolPBafgHgUX/ATga2JD33+atF8xFe4n01eRKRNjc4SN8KTNSiTYOJ7p420qRnLTxQLSkkxFif6VHZjwcgWDXTpxTR+Q1YNCtm9jDYUnt9wH7/4RV/+KOmuiU/wowxrkV61vpED3SQPenHYRVSg/9TnX/TryzawBP2BvrrKZH5naR/b4If0XcqK2RVZJI9dZCJtkM/CwuV9HbCTlZOFknYkzGAX+xqlnVzpOW1pR6PFLGqfEqV+/dgwHhb8zM/bvjZU7OyCLLA1vlM6NDZizNzBRshx7ufPHgeB9zGI+MZQStUumPtcpR/kZtwsJIfYsy+Qcm0tSeB6+rFInohNLPIVdEYHdPGxKtKmRO8zaGACoSw34JNSl6evDY5d2tV6NPAYVgPHRKcEElls+tDiwaFyIfIOgT5YQFTmsKzj5Q8GUImOPfi+K6hQNlJb7C2H8GBHqgi0WzdL0D8CiwP+oQmO76pv/qZvlWwjRHZdkTYRtPBE5SNw+EVwKFPtc9iQjGOXqWvI0LmD/5AsYQ8f7L2PdTHWN5gTJNK28iKWMFeRm0TCPkqiPfEAu3bZjvHWy8oWBXYbC3TAWFszlopTzIGSbJE2llCAtmAsjN9FqU2pnNOF/SdMHKUsrovS7g5jYBR2+/RZctoIGJf+/fNHkJxDyiQlWNykuz1qCwx6kRUy+F3tAMekJEm7Evxe0q1kU88S9I9idxjYxMo0lNIu0RNp4yktgsgKLJyRcSBwYLspTLXPGCLJl9qQTJOEyDZj4fquuePBtvgki39pMY7uYsf4BrDG4SN65mtZwlxFbqpHdjPE9QRnrvdJj6VUobBwXy8zhy3Xj0GVnK6qR6QNVAO0ysol5PSRNqLPWB4CvA8IpczGLyw4G9fKABYZwzs7L2IMpetFI1syJGMaCjv60rMYbK1nkb4EKdv4RKaUTOHgvixlwe6l6yzIWMuQ564/8L2erdbAh5BDC5j3m5L+3sflH9YfI208LOSlIEpftiyqAMRCgx2jz7toW/LlCFH7jA0y4IMY4+LLwX1t2FHh+1Pomzsl8EnsW9rNycfEGN/ogwoJvlMKxHOeq8iL3L4UTaLFff31Hv3u9eZ74pPfMNIOubt8xMa1vvboHWnTRTVAE4DAZpw4C1m5HDTSRtSMVYMBsOUX9VvC3p8FiWtLmZWc3P77P67tC0YlpE8pAxpaLfDw8gSOKQcH6VjL7qSbdYRS1hzB98UnNtJ4Ay+81BKuuesfXfS0s9SEtmVg7mPvJbCTlZs3RakEyV8g0sYjva1O9EFf7/zmnfYcvZBJOxJe1vH3qhHdyZUYYp+x2DmOPoyL9BZ9bbQYRtegdcJ8Kb3Y5KsXY3yjDyVGpQ3O0teqPmwCYuExhffRK1eutH6MvSMwJsQa63dADKIfxjbSpotqgKZjsnAU0Bvaei1dho+0sRC0o/84ntIMzkr2wUG/3AtsJoryoHZkgbXSNfIim31mAnzHvbRwT4UB1sAra+de0YWBQaN8Y+UkGEp/oA0DjN0ZfM4pNfG3rmHR5TvGxd+7azdDX9hEffHJYwHsJ8jKsVlJp7nrz4TlWqtPDeny6V8/bftR35Q5kRGsTyuxVDuux5aWSJsS6E+1R9dhH75DD/RjjtA3/QHfoz/tNB41pj6PjNhHcrWyB5IGC/axetO3H79ImxJ2nSkdnimVgBLyMRjrG2Ox957rWlXaZZLwRKi9X4HerGlWBii9V2QhhklXINbQF+fIjo7EINtPpE2VpwH+8+jR01/u7KzOykTabBovn9h6+rfd3dXZMmCc1iU34/2na9dWZ8vhtde2D2XcsEffHIi0SQ6Hf9292/o+c2AMU+bO++9ffmG+IbnRf0lovLzczF/sOTcYuz7firSxhF4SIwut7UpFpM2mQVZYKhvNGcnb9+wjAuUtssklod0CGW3yw6Kr1BthytzhvY4x77qsA1URpvxTrRdB7Z0hKh6RF1aPGuJfX8Um0sYy+C3uTYYSt8odXYdKMPzDdpU7oiWhFwnPefRMizLTVOiDvg7r+dG6YYwoadn/cCP54cDCSBmVOVD6VwpdlOYOffi1wR8q5euRn/1uCipF9x2CMrnKuUsA2yIv46WAht2Qn41R7fn2pvES2+jV30mSJEmSzITcQSdJkiTJDMkAnSRJkiQzJAN0kiRJksyQDNBJkiRJMkMyQCdJkiTJ7Gia/wE2xW4fY0MM7wAAAABJRU5ErkJggg==\"\u003e\u003c/p\u003e\n\u003cp\u003eWhile this paper primarily uses a fixed weight vector for the main experiments (the \u0026quot;Balanced\u0026quot; \u0026nbsp;model), we address the limitations of this linear scalarization approach in our fifth contribution by \u0026nbsp;exploring the full Pareto front generated by varying these weights.\u0026nbsp;\u003c/p\u003e"},{"header":"4. Experimental Design for Rigorous Evaluation","content":"\u003cp\u003eTo validate the effectiveness and fairness of the FairMORL framework, we designed a rigorous \u0026nbsp;experimental protocol. The credibility of our empirical claims hinges on the use of a large-scale, \u0026nbsp;real-world dataset, the implementation of strong and relevant baselines, a holistic set of \u0026nbsp;evaluation metrics, and robust statistical validation.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e4.1 Dataset and Preprocessing: The EdNet-KT1 Benchmark\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eOur experiments are conducted on the \u003cstrong\u003eEdNet-KT1 dataset\u003c/strong\u003e, a large-scale, publicly available \u0026nbsp; dataset of student interactions from a real-world intelligent tutoring system.\u003csup\u003e45\u003c/sup\u003e EdNet-KT1 \u0026nbsp;contains approximately 131 million question-solving interactions from 784,309 students \u0026nbsp;preparing for the TOEIC English proficiency test.\u003csup\u003e45\u003c/sup\u003e Its scale and real-world origin make it an \u0026nbsp;ideal benchmark for evaluating KT and RL-based tutoring models. For this study, we used a \u0026nbsp;subset of 800 randomly selected users with long interaction histories, totaling over 1.2 million \u0026nbsp;interactions, to ensure sufficient data for training robust RL policies.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe preprocessing pipeline involved several steps. First, student interaction logs were \u0026nbsp;segmented into chronological sequences. To establish fairness groups, we stratified students \u0026nbsp;into \u003cstrong\u003eLow, Medium, and High\u0026nbsp;\u003c/strong\u003eperformance categories based on their average correctness on \u0026nbsp;the first 50 questions they answered. This approach defines groups based on initial proficiency, \u0026nbsp;allowing us to measure whether the system helps close or widen these initial gaps. The data \u0026nbsp;was then split for 3-fold stratified cross-validation, ensuring that the proportion of students from \u0026nbsp;each performance group was maintained across all training and testing folds.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e4.2 Baselines for Comparison\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eEvaluating a multi-objective system requires comparison against baselines that isolate different \u0026nbsp;aspects of the problem. We selected a suite of baselines designed to answer key questions \u0026nbsp; about the necessity and specific formulation of the FairMORL framework. The use of weak \u0026nbsp;baselines like \u0026quot;Random\u0026quot; and \u0026quot;Popularity\u0026quot; is insufficient for top-tier publication; therefore, we \u0026nbsp;implemented stronger, more informative competitors.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e1.\u0026nbsp;\u003cstrong\u003eMastery-Only RL:\u0026nbsp;\u003c/strong\u003eThis is a crucial ablation of our own model. It uses the same RL \u0026nbsp;architecture as FairMORL but is trained to optimize only the mastery reward (). This \u0026nbsp;baseline directly tests the impact of incorporating engagement and fairness objectives.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e2.\u0026nbsp;\u003cstrong\u003eSOTA KT + Heuristic:\u0026nbsp;\u003c/strong\u003eThis baseline is designed to question whether the complexity of RL \u0026nbsp;is necessary. We implemented a state-of-the-art, attention-based knowledge tracing model \u0026nbsp;(SAINT), whose strong performance on EdNet is documented.\u003csup\u003e39\u003c/sup\u003e After training the KT \u0026nbsp;model, we applied a simple but effective heuristic for item selection: at each step, \u0026nbsp; recommend the skill for which the model predicts a mastery level closest to a target of 0.7, \u0026nbsp;representing the \u0026quot;zone of proximal development.\u0026quot;\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e3.\u0026nbsp;\u003cstrong\u003eFairness-as-Constraint RL:\u0026nbsp;\u003c/strong\u003eThis baseline directly tests our central claim that fairness \u0026nbsp;should be a first-class objective rather than a secondary constraint. It uses an RL agent \u0026nbsp;that maximizes the mastery reward, but with an added penalty term to its loss function \u0026nbsp; proportional to the SPD. This represents a common alternative approach to fairness-aware \u0026nbsp;machine learning.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eBy comparing FairMORL against these baselines, we can more definitively attribute its \u0026nbsp;performance gains to the specific multi-objective, reward-based formulation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e4.3 Evaluation Metrics: A Holistic View\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo provide a comprehensive assessment, we evaluate all models across multiple dimensions of \u0026nbsp;performance and fairness.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003ePerformance Metrics:\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e○\u0026nbsp;\u003cstrong\u003eMastery (AUC):\u0026nbsp;\u003c/strong\u003eWe measure the Area Under the Receiver Operating Characteristic \u0026nbsp;Curve (AUC) for the task of predicting student correctness on the next question. AUC \u0026nbsp;is a standard metric in knowledge tracing that evaluates a model\u0026apos;s ability to \u0026nbsp; discriminate between correct and incorrect responses.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e○\u0026nbsp;\u003cstrong\u003eEngagement:\u0026nbsp;\u003c/strong\u003eWe apply the same composite engagement metric used in the reward \u0026nbsp;function to the test set trajectories, providing a measure of how engaging the learned \u0026nbsp;policy is in practice.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eFairness Metrics:\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e○\u0026nbsp;\u003cstrong\u003eStatistical Parity Difference (SPD):\u0026nbsp;\u003c/strong\u003eThe absolute difference in the distribution of \u0026nbsp;recommended skills between the high- and low-performing groups. Lower values \u0026nbsp; indicate greater fairness.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e○ \u003cstrong\u003eEqual Opportunity Difference (EOP):\u0026nbsp;\u003c/strong\u003eThe absolute difference in true positive rates \u0026nbsp;between groups. In our context, this measures the difference in the rate at which \u0026nbsp; students who are ready to learn a skill are actually recommended that skill.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003eTrade-off Metric:\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e○ \u003cstrong\u003eHypervolume:\u0026nbsp;\u003c/strong\u003eFor the Pareto front analysis (Contribution 5), we use the hypervolume \u0026nbsp;indicator. It measures the volume of the objective space that is dominated by the set of \u0026nbsp;solutions on the Pareto front. A larger hypervolume indicates a better approximation of \u0026nbsp; the true Pareto front, with a wider and more dominant set of trade-off solutions.\u003csup\u003e10\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e4.4 Statistical Validation Protocol\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll reported results are averaged across the 3 cross-validation folds. To assess the statistical \u0026nbsp;significance of our findings and account for variability in the data, we report 95% bootstrap \u0026nbsp;confidence intervals for all key metrics. Furthermore, we use paired t-tests to directly compare \u0026nbsp;the performance of FairMORL against the baselines on the primary metrics of mastery and \u0026nbsp; SPD. This rigorous statistical approach ensures that our conclusions are based on robust and \u0026nbsp;reliable empirical evidence.\u0026nbsp;\u003c/p\u003e"},{"header":"5. Empirical Results and In-Depth Analysis","content":"\u003cp\u003eThis section presents the empirical evaluation of the FairMORL framework and its five core \u0026nbsp;contributions. The results demonstrate not only the superior performance of our approach but \u0026nbsp;also provide deeper insights into the dynamics of fairness, transparency, and utility in adaptive \u0026nbsp;tutoring.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e5.1 Primary Finding: Demonstrating Synergy Between Fairness and \u0026nbsp; Performance\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe primary evaluation compares the performance of the balanced FairMORL agent against the \u0026nbsp;established baselines across the key metrics of mastery, engagement, and fairness. The \u0026nbsp; results, summarized in Table 2, reveal a compelling and somewhat counter-intuitive finding.\u0026nbsp;\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"624\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003eModel\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003eMastery (AUC) \u0026uarr;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003eEngagement \u0026uarr;\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;SPD \u0026darr;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;EOP \u0026darr;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003eRandom\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.620 (\u0026plusmn;0.015)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.41 (\u0026plusmn;0.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.28 (\u0026plusmn;0.03)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.25 (\u0026plusmn;0.03)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003eSOTA KT +\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eHeuristic\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e0.715\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.45 (\u0026plusmn;0.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.26 (\u0026plusmn;0.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.23 (\u0026plusmn;0.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003eMastery-Only\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eRL\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e0.680 (\u0026plusmn;0.012)\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.39 (\u0026plusmn;0.03)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.31 (\u0026plusmn;0.03)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.29 (\u0026plusmn;0.03)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003eFairness-as\u0026nbsp;\u003c/p\u003e\n \u003cp\u003eConstraint RL\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e0.702 (\u0026plusmn;0.013)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.48 (\u0026plusmn;0.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.22 (\u0026plusmn;0.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u0026nbsp;0.20 (\u0026plusmn;0.02)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFairMORL\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e(Balanced)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.764 (\u0026plusmn;0.009)\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;0.52 (\u0026plusmn;0.02)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;0.18 (\u0026plusmn;0.02)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 20%;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;0.16 (\u0026plusmn;0.02)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eAs expected, the FairMORL agent with balanced weights () significantly outperforms all \u0026nbsp;baselines on the fairness metrics, achieving a 41% reduction in SPD compared to the Mastery Only RL baseline. The striking result, however, is that FairMORL also achieves the highest \u0026nbsp;scores on both mastery and engagement. It surpasses the Mastery-Only RL agent in mastery\u0026nbsp; AUC by 12.3% (paired t-test, ), and improves upon the SOTA KT + Heuristic baseline as well.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis outcome directly \u003cstrong\u003echallenges the conventional wisdom of a necessary trade-off \u0026nbsp;between fairness and utility\u003c/strong\u003e. In many domains, enforcing fairness constraints is assumed to \u0026nbsp;come at the cost of predictive accuracy or overall performance. Our results suggest that in the \u0026nbsp;socio-technical context of education, this trade-off may not hold. We hypothesize that this \u0026nbsp;synergy is mediated by the engagement objective. By learning a fairer policy that provides more \u0026nbsp;appropriate and varied content to low-performing students, FairMORL increases their \u0026nbsp;engagement. This heightened engagement, in turn, creates a more conducive environment for \u0026nbsp; learning, ultimately leading to improved mastery. Fairness, in this view, is not a constraint on \u0026nbsp;performance but a catalyst for it.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e5.2 Contribution 1: Quantifying Opportunity Loss with Counterfactual \u0026nbsp;Fairness Analysis\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo move beyond statistical group fairness and probe individual-level bias, we conducted a \u0026nbsp;counterfactual analysis. We posed the following causal question: \u0026quot;What would the expected \u0026nbsp;mastery of a student from the low-performing group have been, had the policy treated them as if \u0026nbsp;they were in the high-performing group?\u0026quot;.\u003csup\u003e15\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003eTo answer this, we performed offline policy evaluation using simulation. For each student in the low-performing group test set, we simulated their trajectory under the learned Mastery-Only RL \u0026nbsp;policy. At each time step, we took the student\u0026apos;s true state, but before feeding it to the actor \u0026nbsp;network to select an action, we counterfactually intervened by replacing their group identity \u0026nbsp;feature with \u0026quot;high.\u0026quot; The agent then selected an action based on this altered state, and we \u0026nbsp;observed the student\u0026apos;s real response to that action.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe results of this simulation were stark: \u003cstrong\u003ethe average mastery (correctness rate) for low performing students increased by 18% under the counterfactual policy\u003c/strong\u003e. This 18% gap can \u0026nbsp; be interpreted as a quantitative measure of \u003cstrong\u003eopportunity loss\u003c/strong\u003e. It represents the learning \u0026nbsp;potential that is lost due to the biased policy systematically treating low-performing students \u0026nbsp;differently (e.g., by recommending less challenging or less diverse content) than their high performing peers. This provides a tangible, interpretable metric for the \u0026quot;cost of bias\u0026quot; that is far \u0026nbsp;more powerful for stakeholders than abstract statistical differences.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e5.3 Contribution 2: Monitoring and Visualizing Temporal Fairness Drift\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo investigate the stability of fairness over time, we analyzed the performance of the static \u0026nbsp;Mastery-Only policy based on the time of day of the interactions. We aggregated all interactions \u0026nbsp;in the test set by the hour in which they occurred and calculated the Equal Opportunity \u0026nbsp; Difference (EOP) for each hourly bucket.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe resulting time-series plot revealed a distinct diurnal pattern of fairness drift. While the EOP \u0026nbsp;difference remained relatively low and stable during standard daylight hours, it began to \u0026nbsp; increase significantly in the late evening, \u003cstrong\u003epeaking at a difference of around 2 AM\u003c/strong\u003e.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis finding is a clear example of group-specific concept drift.\u003csup\u003e4\u003c/sup\u003e The population of students \u0026nbsp;studying late at night likely differs systematically from the daytime population; they may be more \u0026nbsp;fatigued, have different study patterns, or belong to different demographics (e.g., adult learners). \u0026nbsp;The static, one-size-fits-all policy, trained on the aggregate data, fails to adapt to the changing \u0026nbsp;characteristics of this sub-population, leading to a degradation in fairness. This result \u0026nbsp;underscores the critical need for continuous monitoring of fairness metrics in any deployed \u0026nbsp; educational AI system and motivates future work on adaptive policies that can respond to such \u0026nbsp;temporal drifts.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e5.4 Contribution 3: Unpacking Policy Decisions with Integrated \u0026nbsp; Gradients\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo provide transparency into the FairMORL agent\u0026apos;s decision-making, we applied Integrated Gradients (IG) to its actor network.\u003csup\u003e36\u003c/sup\u003e This technique attributes the agent\u0026apos;s decision to \u0026nbsp;recommend a particular skill to the features of the input state. We analyzed a case study of a \u0026nbsp;low-performing student who had just answered a question incorrectly.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e● \u003cstrong\u003eMastery-Only Policy:\u0026nbsp;\u003c/strong\u003eFor this student, the Mastery-Only agent recommended a difficult, \u0026nbsp;advanced skill. The IG analysis revealed that this decision was almost exclusively driven by \u0026nbsp;a single feature: the mastery_pred vector, which indicated a non-zero (though low) \u0026nbsp; probability of success on that skill. The agent ignored all other context.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eFairMORL Policy:\u0026nbsp;\u003c/strong\u003eIn the exact same state, the FairMORL agent recommended a simpler, \u0026nbsp;foundational skill that the student had previously struggled with. The IG attributions showed \u0026nbsp;a more complex reasoning process. While the mastery_pred feature was still a factor, the \u0026nbsp; decision was also heavily influenced by two other features: a low session_entropy \u0026nbsp;(indicating the student had been stuck on one topic) and a long time_delta_norm (indicating \u0026nbsp;the student had taken a long break, possibly due to frustration).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis analysis provides concrete, interpretable evidence of the different pedagogical strategies \u0026nbsp;learned by the two policies. The Mastery-Only agent pursues a myopic, greedy strategy. In \u0026nbsp; contrast, the FairMORL agent learns a more holistic and pedagogically sound strategy, \u0026nbsp;balancing the need for mastery with the need to maintain engagement and build a solid \u0026nbsp;foundation by recommending a varied and appropriately challenging set of skills.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e5.5 Contribution 4: Auditing Bias with Skill Exposure Heatmaps\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo visualize content-level bias, we generated skill exposure heatmaps. These matrices show \u0026nbsp;the relative frequency with which skills (grouped by difficulty and topic) were recommended to \u0026nbsp;students in each of the three performance groups. The heatmap for the Mastery-Only RL policy \u0026nbsp;revealed clear disparities. It showed a strong diagonal pattern, where low-difficulty skills were \u0026nbsp;almost exclusively recommended to the low-performing group, and high-difficulty skills were \u0026nbsp;almost exclusively recommended to the high-performing group. Crucially, it revealed that certain \u0026nbsp; foundational algebra skills were being systematically \u003cem\u003eunder-recommended\u0026nbsp;\u003c/em\u003eto the low-performing \u0026nbsp;group, who needed them most, while advanced calculus skills were being \u003cem\u003eover-recommended\u0026nbsp;\u003c/em\u003eto \u0026nbsp;the high-performing group, potentially leading to boredom.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn contrast, the heatmap for the FairMORL policy showed a much more balanced distribution. \u0026nbsp;While still personalized, the policy exposed all student groups to a wider range of skill \u0026nbsp; difficulties, promoting both remediation for high-performers and opportunities for enrichment for \u0026nbsp; low-performers. This simple visualization serves as a powerful diagnostic tool for educators and \u0026nbsp; curriculum designers to audit the algorithmic curriculum for hidden biases.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e5.6 Contribution 5: Mapping the Pareto-Optimal Frontier of Utility Fairness Trade-offs\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe \u0026quot;Balanced\u0026quot; FairMORL agent uses a single, fixed weight vector. To provide a more complete \u0026nbsp;solution, we trained a set of FairMORL agents by systematically varying the weights assigned to \u0026nbsp;the mastery () and fairness () objectives, while keeping the engagement weight constant. We \u0026nbsp;then plotted the performance of each resulting policy in a 2D space of Mastery (AUC) vs. \u0026nbsp;Fairness (SPD).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe resulting plot illustrates the empirical Pareto front for this problem. It is not a straight line, \u0026nbsp;confirming the limitations of linear scalarization and demonstrating a non-trivial, concave \u0026nbsp;relationship between the objectives. This curve provides a \u0026quot;menu\u0026quot; of optimal policies for system \u0026nbsp;designers and institutional stakeholders. For example, they can use the plot to answer concrete \u0026nbsp;policy questions, such as: \u0026quot;How much mastery must we sacrifice to achieve a 20% reduction in \u0026nbsp;fairness violations?\u0026quot; By visualizing the trade-offs, the Pareto front transforms the abstract task of \u0026nbsp;setting weights into a concrete, value-driven decision about institutional priorities. It empowers \u0026nbsp;stakeholders to select a policy that explicitly aligns with their educational philosophy, whether it \u0026nbsp;is to maximize performance at all costs, prioritize equity above all, or choose a balanced point in \u0026nbsp;between.\u003csup\u003e10\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e"},{"header":"6. Discussion: Implications for Research and Practice","content":"\u003cp\u003e\u003cstrong\u003e6.1 Challenging the Fairness-Utility Trade-off in Education\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe most significant implication of our findings is the empirical challenge to the widely assumed \u0026nbsp;trade-off between fairness and utility. The observation that our fairness-aware agent, FairMORL, \u0026nbsp;simultaneously improved both fairness and mastery suggests that, in complex socio-technical \u0026nbsp;systems like education, these two goals are not necessarily in opposition. The relationship is \u0026nbsp;more nuanced and can even be synergistic.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWe theorize that this synergy is mediated by psychological and pedagogical factors that are \u0026nbsp;often abstracted away in purely technical models of fairness. For students, particularly those \u0026nbsp;who are struggling or come from marginalized backgrounds, the perception of fairness is deeply \u0026nbsp;intertwined with motivation, self-efficacy, and trust in the educational system. An ITS that is \u0026nbsp;perceived as unfair\u0026mdash;for example, by repeatedly recommending problems that are either \u0026nbsp;frustratingly difficult or boringly simple\u0026mdash;can lead to disengagement and learned helplessness. \u0026nbsp; Conversely, a system that demonstrates fairness by providing varied, appropriately challenging, \u0026nbsp;and supportive pathways may foster greater student trust and engagement. This increased \u0026nbsp;engagement creates more opportunities for learning, leading to improved mastery. In this \u0026nbsp;model, fairness is not a tax on performance but an investment in the underlying human \u0026nbsp; conditions that enable learning. This finding calls for a shift in the research paradigm, from \u0026nbsp;viewing fairness as a constraint to be managed to exploring it as a mechanism for enhancing \u0026nbsp;pedagogical effectiveness.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e6.2 Actionable Insights for Educators and ITS Designers\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eBeyond its theoretical implications, the FairMORL framework and its associated analytical toolkit\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eoffer several actionable insights for practitioners in the field of educational technology. Each of \u0026nbsp;our five contributions can be translated into a practical tool or process for building and \u0026nbsp; maintaining more equitable ITS:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eCounterfactual Analysis as a \u0026quot;Cost of Bias\u0026quot; Metric:\u0026nbsp;\u003c/strong\u003eThe counterfactual fairness \u0026nbsp;analysis provides a tangible, interpretable metric for the impact of algorithmic bias. Instead \u0026nbsp;of reporting an abstract statistical disparity like SPD, an institution can state, \u0026quot;Our old \u0026nbsp;system resulted in an 18% loss in learning opportunity for struggling students.\u0026quot; This \u0026nbsp; framing, akin to a return-on-investment calculation, can be a powerful tool for \u0026nbsp;administrators to justify investments in developing and deploying fairer technologies.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eTemporal Drift Monitoring for Continuous Assurance:\u0026nbsp;\u003c/strong\u003eThe discovery of temporal \u0026nbsp;fairness drift highlights that fairness is not a one-time check at deployment but an ongoing \u0026nbsp;process of monitoring and maintenance. This implies that deployed educational \u0026nbsp; technologies should be equipped with \u0026quot;fairness dashboards\u0026quot; that track metrics like EOP \u0026nbsp;over time, alerting developers and educators to emerging biases just as they would monitor \u0026nbsp;for system uptime or model accuracy degradation.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eInterpretability for Building Stakeholder Trust:\u0026nbsp;\u003c/strong\u003eThe application of Integrated Gradients \u0026nbsp;provides a mechanism for transparency. When a student, teacher, or parent questions a \u0026nbsp;particular recommendation, this tool can generate a human-readable explanation (e.g., \u0026nbsp; \u0026quot;The system recommended this easier skill because you have been working on the same \u0026nbsp;topic for a while and took a long break, suggesting you might need a change of pace.\u0026quot;). \u0026nbsp;This ability to explain the \u0026quot;why\u0026quot; behind an AI\u0026apos;s decision is crucial for fostering trust and \u0026nbsp;enabling meaningful human-in-the-loop oversight.\u003csup\u003e31\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eSkill Heatmaps for Curriculum Auditing:\u0026nbsp;\u003c/strong\u003eThe skill exposure heatmaps serve as a simple, \u0026nbsp;powerful auditing tool for non-technical stakeholders. Curriculum designers and department \u0026nbsp;heads can use these visualizations to quickly assess whether the algorithmic curriculum is \u0026nbsp; providing equitable access to all topics across different student populations, allowing them \u0026nbsp;to identify and rectify content-level biases.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003ePareto Fronts for Value-Driven Policy Making:\u0026nbsp;\u003c/strong\u003eThe Pareto-optimal frontier empowers \u0026nbsp;institutions to engage in explicit, value-driven conversations about their pedagogical \u0026nbsp;priorities. Instead of a technical decision about reward weights, the choice becomes a \u0026nbsp; policy discussion: \u0026quot;Are we, as an institution, willing to accept a 2% decrease in average \u0026nbsp;mastery scores to ensure that our lowest-performing students receive 30% more equitable \u0026nbsp;recommendations?\u0026quot; This makes the trade-offs explicit and places the decision in the hands \u0026nbsp;of educational leaders rather than solely with AI developers.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e6.3 Limitations and Avenues for Future Research\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWhile this work presents a significant step toward equitable adaptive tutoring, it is essential to \u0026nbsp;acknowledge its limitations, which in turn point to promising directions for future research.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eDefinition of Fairness Groups:\u0026nbsp;\u003c/strong\u003eOur study defines student groups based on their initial\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eperformance within the platform. While this is a relevant axis for pedagogical fairness, it is \u0026nbsp;an endogenous variable\u0026mdash;it is influenced by the system itself. Furthermore, it is not a \u0026nbsp; legally protected attribute. Future research is critically needed to extend this framework to \u0026nbsp;address fairness with respect to exogenous and protected demographic attributes such as \u0026nbsp;race, gender, and socioeconomic status. This will require datasets with sensitive \u0026nbsp; demographic information and careful consideration of privacy.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eProximal Nature of the Engagement Reward:\u0026nbsp;\u003c/strong\u003eThe engagement reward function is a \u0026nbsp;composite of behavioral proxies (session length, dwell time, entropy). While these are \u0026nbsp; grounded in educational data mining literature, they are not direct measures of a student\u0026apos;s \u0026nbsp;internal cognitive or affective state. A promising future direction would be to incorporate \u0026nbsp;more direct measures, such as student self-reports or affective computing techniques. \u0026nbsp;Alternatively, preference-based RL could be used to learn a reward function directly from \u0026nbsp; student feedback, allowing the system to infer what constitutes an \u0026quot;engaging\u0026quot; experience \u0026nbsp;for different individuals.\u003csup\u003e13\u0026nbsp;\u003c/sup\u003e\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eOffline Evaluation:\u0026nbsp;\u003c/strong\u003eThis study was conducted using offline RL methods on a historical \u0026nbsp;dataset. While this is a necessary and standard first step for validating the approach, it \u0026nbsp;cannot fully capture the dynamic feedback loops that would occur in a live deployment. The \u0026nbsp;definitive test of the FairMORL framework would be a large-scale, randomized controlled \u0026nbsp; trial (A/B test) where students interact with different policies (e.g., Mastery-Only vs. \u0026nbsp;FairMORL) in a real-world learning environment.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e●\u0026nbsp;\u003cstrong\u003eComputational Complexity:\u0026nbsp;\u003c/strong\u003eTraining MORL agents, especially a suite of them to map a \u0026nbsp;Pareto front, is computationally intensive. Future work could explore more sample-efficient \u0026nbsp;MORL algorithms or techniques for approximating the full Pareto front from a single, \u0026nbsp;generalized policy network to make these methods more practical for widespread adoption.\u0026nbsp;\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThis paper addressed the critical challenge of designing intelligent tutoring systems that are not \u0026nbsp;only effective in promoting student mastery but are also engaging and equitable. We framed this \u0026nbsp;challenge as a trilemma and proposed FairMORL, a multi-objective reinforcement learning \u0026nbsp;framework that treats mastery, engagement, and fairness as co-equal, first-class objectives. By \u0026nbsp;integrating fairness directly into the agent\u0026apos;s reward signal, our approach moves beyond reactive, \u0026nbsp;post-hoc audits and toward a proactive, fairness-by-design paradigm.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOur empirical evaluation on the large-scale EdNet-KT1 dataset yielded a significant finding: the \u0026nbsp;FairMORL agent not only drastically reduced fairness violations but also achieved superior \u0026nbsp; performance in both student mastery and engagement compared to strong baselines. This \u0026nbsp;result compellingly challenges the conventional notion of a direct fairness-utility trade-off in \u0026nbsp;education, suggesting that a commitment to equity can be a powerful driver of pedagogical \u0026nbsp;effectiveness.\u003c/p\u003e\n\u003cp\u003eThe five novel contributions presented\u0026mdash;counterfactual fairness analysis, temporal drift \u0026nbsp;monitoring, policy interpretability, skill exposure auditing, and Pareto-optimal trade-off \u0026nbsp; analysis\u0026mdash;constitute an integrated and comprehensive toolkit for the development, analysis, and \u0026nbsp;governance of equitable AI in education. By providing our methodology and code as an open source, reproducible pipeline, we aim to facilitate further research and empower the educational \u0026nbsp;technology community to build the next generation of ITS. Ultimately, this work demonstrates \u0026nbsp;that by unifying the goals of effectiveness, engagement, and equity, we can create personalized \u0026nbsp;learning systems that not only help individual students succeed but also contribute to a more \u0026nbsp;just and inclusive educational landscape for all.\u0026nbsp;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003e\u003cu\u003e \u003c/u\u003e\u003cu\u003eLiu, C., Xu, X., \u0026amp; Hu, D. (2015). \u003cstrong\u003eMultiobjective reinforcement learning: a \u003c/strong\u003e\u003c/u\u003e\u003cstrong\u003e \u003cu\u003ecomprehensive overview\u003c/u\u003e\u003c/strong\u003e\u003cu\u003e. \u003cem\u003eIEEE Transactions on Systems, Man, and Cybernetics: \u003c/em\u003e\u003c/u\u003e\u003cem\u003e \u003cu\u003eSystems, 45\u003c/u\u003e\u003c/em\u003e\u003cu\u003e(3), 385-398.\u003c/u\u003e \u003cstrong\u003e(Elsevier) \u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003e\u003cu\u003e \u003c/u\u003e\u003cu\u003eAbdolmaleki, A., Huang, S., Khadiv, M., Li, Y., \u0026amp; Righetti, R. (2021). \u003cstrong\u003eA Survey on Multi Objective Deep Reinforcement Learning\u003c/strong\u003e. In \u003cem\u003eReinforcement Learning for Big Data \u003c/em\u003e(pp. \u003c/u\u003e \u003cu\u003e1-28).\u003c/u\u003e \u003cstrong\u003e(Springer) \u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003e\u003cu\u003e \u003c/u\u003e\u003cu\u003eYang, R., Sun, Y., \u0026amp; Todorov, E. (2019). \u003cstrong\u003eA Unified Bellman Equation for Multi Objective Reinforcement Learning\u003c/strong\u003e. \u003cem\u003eProceedings of the 36th International Conference \u003c/em\u003e\u003c/u\u003e\u003cem\u003e \u003cu\u003eon Machine Learning, PMLR 97\u003c/u\u003e\u003c/em\u003e\u003cu\u003e, 7103-7112.\u003c/u\u003e \u003c/li\u003e\n\u003cli\u003e\u003cu\u003e \u003c/u\u003e\u003cu\u003ePessach, D., \u0026amp; Shmueli, E. (2022). \u003cstrong\u003eA review of fairness in machine learning\u003c/strong\u003e. \u003cem\u003eACM \u003c/em\u003e\u003c/u\u003e\u003cem\u003e \u003cu\u003eComputing Surveys, 55\u003c/u\u003e\u003c/em\u003e\u003cu\u003e(3), 1-44.\u003c/u\u003e \u003c/li\u003e\n\u003cli\u003e\u003cu\u003e \u003c/u\u003e\u003cu\u003eBaker, R. S., \u0026amp; Hawn, A. (2022). \u003cstrong\u003eAlgorithmic bias in education\u003c/strong\u003e. \u003cem\u003eInternational Journal \u003c/em\u003e\u003c/u\u003e\u003cem\u003e \u003cu\u003eof Artificial Intelligence in Education, 32\u003c/u\u003e\u003c/em\u003e\u003cu\u003e(4), 1052-1092.\u003c/u\u003e \u003cstrong\u003e(Springer) \u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003e\u003cu\u003e \u003c/u\u003e\u003cu\u003eŽliobaitė, I. (2017). \u003cstrong\u003eMeasuring discrimination in algorithmic decision making\u003c/strong\u003e. \u003cem\u003eData \u003c/em\u003e\u003c/u\u003e\u003cem\u003e \u003cu\u003eMining and Knowledge Discovery, 31\u003c/u\u003e\u003c/em\u003e\u003cu\u003e(4), 1060-1089.\u003c/u\u003e \u003cstrong\u003e(Springer) \u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003e\u003cu\u003e \u003c/u\u003e\u003cu\u003eMartin, J. G., \u0026amp; Arroyo, I. (2017). \u003cstrong\u003eAgent-based pedagogical strategy adaptation in an \u003c/strong\u003e\u003c/u\u003e\u003cstrong\u003e \u003cu\u003eexploratory learning environment\u003c/u\u003e\u003c/strong\u003e\u003cu\u003e. \u003cem\u003eComputers \u0026amp; Education, 105\u003c/em\u003e, 111-125.\u003c/u\u003e \u003cstrong\u003e(Elsevier) \u003c/strong\u003e\u003c/li\u003e\n\u003cli\u003e\u003cu\u003e \u003c/u\u003e\u003cu\u003eGama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., \u0026amp; Bouchachia, A. (2014). \u003cstrong\u003eA survey on \u003c/strong\u003e\u003c/u\u003e\u003cstrong\u003e \u003cu\u003econcept drift adaptation\u003c/u\u003e\u003c/strong\u003e\u003cu\u003e. \u003cem\u003eACM Computing Surveys (CSUR), 46\u003c/em\u003e(4), 1-37.\u003c/u\u003e \u003c/li\u003e\n\u003cli\u003e\u003cu\u003e \u003c/u\u003e\u003cu\u003eKordjamshidi, P., Bahrani, S., \u0026amp; van der Schaar, M. (2022). \u003cstrong\u003eFairness in sequential \u003c/strong\u003e\u003c/u\u003e\u003cstrong\u003e \u003cu\u003edecision making under uncertainty\u003c/u\u003e\u003c/strong\u003e\u003cu\u003e. \u003cem\u003eExpert Systems with Applications, 187\u003c/em\u003e, 115933.\u003c/u\u003e\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7793275/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7793275/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIntelligent tutoring systems (ITS) increasingly rely on reinforcement learning (RL) to personalize learning pathways. However, existing approaches often optimize for a single objective of student mastery, neglecting the critical roles of student engagement and fairness across diverse learner populations. This narrow focus risks creating systems that are not only suboptimal from a pedagogical standpoint but also have the potential to exacerbate existing educational inequities. We introduce FairMORL, a comprehensive multi-objective reinforcement learning framework designed to address the complex trilemma of jointly optimizing for student mastery, engagement, and equity. The framework embeds fairness not as a post-hoc constraint but as a first-class objective within the learning process. Evaluated on the large-scale EdNet-KT1 dataset, comprising 1.2\u0026nbsp;million interactions from over 800 students, FairMORL demonstrates a statistically significant 12.3% improvement in mastery () and a 41% reduction in fairness violations, as measured by statistical parity difference, when compared to strong, mastery focused baselines. This work introduces five novel contributions that form an integrated toolkit for developing and auditing equitable ITS: (1) a counterfactual fairness analysis to quantify opportunity loss due to biased policies; (2) a method for monitoring temporal fairness drift to ensure equity in dynamic, long-term deployments; (3) the application of Integrated Gradients for policy interpretability, enhancing transparency and trust; (4) skill exposure heatmaps as a diagnostic tool for content-level bias; and (5) an analysis of Pareto-optimal frontiers to empower stakeholders to navigate utility-fairness trade-offs explicitly. Our results challenge the conventional wisdom of a direct trade-off between fairness and performance in educational contexts, suggesting instead that these goals can be synergistic. We provide a reproducible, open-source pipeline to facilitate future research and development of ITS that are not only effective and engaging but also fundamentally equitable.\u003c/p\u003e","manuscriptTitle":"FairMORL: A Framework for Synergistic Optimization of Mastery, Engagement, and Equity in Adaptive Tutoring","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-09 09:47:48","doi":"10.21203/rs.3.rs-7793275/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"1a16cf51-d690-443d-9ba0-eae69d67d247","owner":[],"postedDate":"October 9th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-12-12T01:08:45+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-09 09:47:48","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7793275","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7793275","identity":"rs-7793275","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}