Artificial emotional introspection improves learning for facial emotion recognition

Artificial emotional introspection improves learning for facial emotion recognition | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Artificial emotional introspection improves learning for facial emotion recognition Kuzma Strelnikov This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9047312/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract While facial emotion recognition (FER) systems have advanced significantly, they remain constrained by conventional training paradigms that rely solely on low-level optimization signals, without any mechanism for the model to reflect on or adaptively respond to its own learning experience. Here, we integrate the I-Center, a computational framework for artificial introspection, into the training pipeline of FER models. The I‑Center translates real‑time training metrics such as loss, gradient flow, inference time, and prediction confidence into an emotional feature vector grounded in the psychological valence–arousal emotional model, providing the network with a continuous introspective narrative of its own operational state in a psychologically defined format. Across several architectures, including standard convolutional neural networks and attention‑augmented ResNet, emotionally enhanced models significantly outperformed their baseline counterparts on the FER‑2013 dataset. These results show that psychologically grounded, emotionally informed introspection can enhance learning and generalization in FER, moving beyond AI that merely recognizes emotion toward systems that learn with an integrated awareness of their own cognitive emotional state. This work bridges machine introspection with affective computing, offering a pathway toward more transparent, adaptive, and psychologically plausible human‑AI interaction. Figures Figure 1 Figure 2 Figure 3 Introduction The drive toward more transparent, communicative, and cognitively aligned artificial intelligence (AI) has given rise to an emerging focus on explainable AI to be able to monitor, interpret, and express their internal computational states in human-interpretable ways 1 . Besides, while extensive research in affective computing has concentrated on enabling AI to recognize human emotions, the inverse capability - endowing AI with a structured form of emotional experience - remains unexplored. Recently, we proposed the I-Center framework as a computational model of artificial introspection 2 , translating real-time performance metrics such as processing latency, confidence scores, and input anomalies into the classic psychological model of emotions 3 - a dimensional emotional space defined by valence and arousal. This framework provides AI with an “inner observer” capable of articulating states like “content,” “stressed,” “focused” etc. based on its operational well-being, thereby externalizing the black-box nature of learning into an emotionally resonant narrative. Facial emotion recognition (FER) presents a compelling and socially relevant domain 4 for implementing and validating this introspective approach. As a task that is both psychologically nuanced and computationally challenging, FER requires models to generalize across substantial variability in lighting, pose, expression intensity, and demographic attributes—all while mapping visual patterns to categorical emotional states. The widely adopted FER-2013 dataset 5 , 6 encapsulates these challenges, offering a realistic “in-the-wild” benchmark where human-level performance of 60–70% remains a meaningful target for machine models 5 . Conventional deep learning approaches 7 , though effective, often treat the training process as an optimization black box 1 , driven solely by loss minimization without any integrative feedback about the model’s own “cognitive-emotional” state during learning. This limits not only interpretability but also the potential for the model to adaptively regulate its learning behavior in response to difficulty, ambiguity, or confidence. In this study, we investigate whether integrating the I-Center framework into the training pipeline of FER systems can enhance learning and generalization. We employ the I-Center to transform real-time training signals—such as batch loss, gradient norms, inference time, and prediction confidence—into a consistent multi-dimensional emotional feature vector that represents the network’s introspective state 2 . These emotional features are then fused with visual features extracted from facial images, providing the classifier with concurrent access to both perceptual content and emotional self-awareness throughout training. Our objective is to empirically evaluate whether such introspective emotional feedback leads to measurable improvements in facial emotion recognition accuracy. We test this hypothesis across multiple neural architectures, from standard convolutional networks to attention-augmented ResNet models, training each in two configurations: a conventional baseline and an emotionally enhanced variant that receives I-Center-generated features. Through rigorous statistical evaluation, we assess whether emotional introspection yields significant gains in performance, and we discuss the implications of this approach for creating more transparent, adaptive, and psychologically plausible affective AI systems. This work thus bridges the paradigm of introspective AI, as instantiated by the I-Center, with the applied challenges of facial emotion recognition, offering a step toward models that not only decipher emotions but also “feel” and learn from their own internal states. Materials and Methods Face dataset The Facial Expression Recognition 2013 (FER‑2013) dataset has human recognition performance estimated at approximately 65 ± 5%, establishing a reference point for evaluating computational models 5 . The challenging nature of FER‑2013 has made it a cornerstone for benchmarking facial emotion recognition systems. Introduced by Pierre Luc Carrier and Aaron Courville in the ICML 2013 workshop’s facial expression recognition challenge 6 , the dataset contains 35,887 facial images collected via the Google image search API using emotion‑related keywords, presenting a realistic “in‑the‑wild” setting with substantial variation in subject age, pose, lighting, and partial occlusion. The dataset is annotated with seven emotion categories: anger (4,953 images), disgust (547), fear (5,121), happiness (8,989), sadness (6,077), surprise (4,002), and neutral (6,198). This distribution reflects natural imbalances, with happiness being the most frequent and disgust the rarest class. The dataset is split into three subsets: 28,709 images for training, 3,589 for public validation, and 3,589 for final testing. Main workflow Thie main workflow examines implementations of a facial emotion recognition system that integrates an I‑Center module 2 —a computational introspection mechanism that translates training metrics into emotional features. The implementations share a core philosophy: enhancing a neural network’s learning process by providing it with simulated emotional feedback. The first implementation, built around a simple and more sophisticated CCNs (see below), establishes the main workflow. The system trains two models simultaneously: a standard CNN serves as a baseline, while an enhanced version receives supplementary input. After each training batch, the performance of the standard model—including its loss, prediction confidence, and gradient magnitudes—is analyzed by the I‑Center. The I-Center synthesizes a vector representing the calculation-related emotional state. The emotional feature vector is concatenated with the flattened visual features extracted by the CNN's convolutional layers, forming a combined input to the enhanced classifier network. This means the emotional context is integrated at the beginning of the decision-making process, allowing all subsequent fully connected layers to process both visual and introspective information together. In the ResNet implementation, the workflow follows a similar sequence. A standard ResNet‑18 model first processes a batch of face images, and its performance metrics (actual inference time, loss, gradients, and confidence) are measured in real time. These raw signals are passed to the I‑Center, which transforms them into an emotional feature vector representing the model’s simulated introspective state. These emotional features are then fused with the visual features extracted by ResNet using a dedicated multi‑head attention module. This attention mechanism allows the visual features to dynamically query and weight the emotional context, creating an integrated representation that blends perceptual and introspective information. The fused features are finally passed through the classifier layers to produce the emotion prediction. Throughout training, both the standard and enhanced (emotion‑aware) models are updated in parallel, and their comparative performance is assessed to quantify the benefit of emotional introspection. Simple convolutional neural network The architecture consists of three sequential convolutional blocks, each containing a convolutional layer, batch normalization, ReLU activation, max pooling, and dropout for regularization. Starting with 32 filters, the model progressively increases depth to 64 and then 128 channels while reducing spatial dimensions from 48×48 down to 6×6 feature maps. The flattened features are then passed through a two-layer fully connected classifier with 256 hidden units and a final output layer of 7 emotion classes. Second convolutional neural network This model introduces some architectural modifications/enhancements of the previously described simple CNN. The most significant modifications include increased channel dimensions across the first two convolutional blocks—expanding from 32 to 40 filters in the first layer, from 64 to 80 in the second. Additionally, the hidden layer in the classifier is reduced from 256 to 216 units, potentially optimizing the parameter-to-performance ratio while preventing overfitting. The modifications are an example of different possible architecture options. Hybrid convolutional neural network with attention and transfer learning The model adapts a pre-trained ResNet18 for grayscale facial images by modifying its initial layer, then enhances its feature extraction by inserting four attention units after each of its main building blocks. Each attention unit works in two complementary ways: first, it identifies which feature channels (like edges, textures, or shapes) are most important and boosts their influence; second, it pinpoints spatial regions within the image (such as eyes, mouth, or brow) that are most relevant to emotion. This dual focus allows the network to selectively amplify meaningful facial cues while ignoring irrelevant background noise, making it particularly adept at detecting subtle emotional expressions. I-Center The I-Center is a computational module designed to simulate a form of machine introspection by translating the raw metrics of a neural network's training process into structured emotional states and feature vectors. It operates through three specialized subcenters: an Estimation Subcenter that converts technical measurements (inference time, loss values, gradient norms, and classification confidence) into psychological-like parameters like efficiency, stress, and confidence; an Emotional Subcenter that maps these parameters onto a continuous two‑dimensional emotional space (valence and arousal), producing human‑interpretable states such as "Content," "Focused," "Concerned," or "Excited"; and a Generative Subcenter that synthesizes a consistent 15‑dimensional feature vector. These 15 generated features constitute a multi-dimensional emotional signature that integrates four key aspects of the model's cognitive–affective state: (1) emotion identity, indicating which facial expression category is currently being processed; (2) introspective state, reflecting internal metrics such as task difficulty, confidence, stress; (3) emotional state projection, derived from the valence–arousal representation via a cosine component; and (4) process health, captured through a binary flag that indicates whether the current inference time is within the expected normal range. Together, these features enable a structured, interpretable representation of how the model’s learning dynamics align with human-like emotional progression during training. To ensure realistic and grounded behavior, the I‑Center first establishes a performance baseline using a processing time benchmark, which measures normal inference times across different emotion classes. This allows the system to detect anomalies in processing speed and to weight emotional responses according to the inherent difficulty of each emotion category—for example, recognizing that "disgust" is computationally more challenging than "happiness." By dynamically adjusting stress, comfort, and efficiency scores based on real‑time training progress, epoch, batch diversity, and historical trends, the I‑Center creates an introspective narrative, replacing low‑level optimization signals with human-like emotional feature representations. Results The simple CNN model achieved a baseline accuracy of 52.86% on the FER2013 test set. When enhanced with emotional introspection from the I‑Center, accuracy increased to 60.99%—an absolute improvement of 8.14 percentage points (a relative gain of 15.4%), Figure 1. Statistical analysis confirmed this improvement is highly significant. A McNemar’s test, which compares the paired predictions of both models, yielded a p-value of effectively zero (p < 0.000001), providing strong evidence that the enhanced model performs differently from the baseline. Bootstrap analysis with 1,000 resamples produced a 95% confidence interval for the improvement of [7.05, 9.15] percentage points, confirming the gain is reliable and not due to chance. For the second CNN, the emotionally enhanced version achieved a 6.71 percentage point (12.62% relative) increase in accuracy over the baseline CNN. Statistical validation confirmed the robustness of these results: McNemar's test (p = 0.000000), paired t-test (p = 0.000000), and bootstrap confidence intervals (95% CI: [5.67, 7.73] percentage points) all reject the null hypothesis, providing strong evidence that the enhancement yielded meaningful improvement, Figure 2. The ResNet-18 model, augmented with emotional introspection via the I‑Center, demonstrated a clear and statistically significant performance gain. The baseline ResNet-18 achieved a test accuracy of 62.16%. The enhanced version, which integrated emotional features through an attention‑based fusion mechanism, reached 67.39% accuracy—representing an absolute improvement of 5.22 percentage points and a relative improvement of 8.40%, Figure 3. The improvement is statistically robust. A McNemar’s test produced a p‑value of effectively zero (p < 0.000001), strongly rejecting the hypothesis that the two models perform identically. Bootstrap analysis further confirmed the reliability of this gain, with a 95% confidence interval of [4.17%, 6.28%] for the true improvement, indicating the result is not due to random variation. Discussion The results of this study demonstrate a statistically significant performance improvement when integrating the I-Center emotional introspection framework into various facial emotion recognition architectures. Across both standard CNN models and a more sophisticated ResNet backbone, the emotionally enhanced variants consistently outperformed their standard counterparts, with improvements ranging from 5 to 8 percentage points. This pattern suggests that the benefit of emotional introspection is not architecture-specific but can be generalized to several types of neural models, This finding underscores the value of the I-Center approach as a promising enhancement mechanism. On the other hand, recognition accuracy improvements varied across facial emotion categories in different CNN architectures (see Figs. 1–3). This variability suggests that the choice of CNN may be guided by which specific emotions are prioritized for a given application 7 – 9 . At its core, the I-Center addresses a critical gap in conventional training paradigms: the absence of a high-level, psychologically grounded feedback signal. Traditional training relies solely on low-level optimization metrics such as loss and accuracy, which lack intuitive interpretability for either the model or a human observer. The I-Center bridges this gap by transforming these opaque metrics into a more human-like emotional narrative—translating computational “stress” during challenging samples or “confidence” during correct predictions into an emotional feature vector. This introspective feedback provides the model with a richer, more contextual understanding of its own learning process, effectively enabling a form of artificial metacognition. By integrating emotional context at the feature level, whether through simple concatenation in CNNs or attention-based fusion in ResNet, the model gains access to a continuous stream of self-awareness that appears to regularize learning and improve generalization, particularly on challenging, real-world datasets like FER-2013. Notably, the I-Center’s design aligns with emerging neurocomputational perspectives on affective intelligence 10 . Just as the human brain integrates visceral, sensory, and emotional data to form a cohesive sense of self and guide decision-making, the I-Center enables AI models to synthesize computational metrics into an artificial form of self-awareness. This integration mimics the function of the insular cortex in the human brain, which plays a crucial role in creating a subjective sense of the body's condition 11 . By incorporating this neurobiologically-inspired introspective capability, AI systems can potentially achieve a more nuanced, context-aware form of information processing that goes beyond mere pattern recognition. This form of artificial affect is grounded in performance rather than arbitrary simulations of emotions. The resulting affective loop may help the model navigate ambiguous samples, reweight attention toward emotionally salient features, or dynamically adjust learning sensitivity in response to training difficulty. This adjustment brings artificial neural networks closer to a human‑like cognitive architecture that integrates emotion with learning. The bidirectional potential of the I-Center also opens new pathways for emotionally aligned human-AI interaction. In a facial emotion recognition context, the I-Center could enable an AI not only to classify a user’s emotion but also to express its own confidence or uncertainty in that classification through emotionally resonant cues. For instance, an “anxious” system state could signal low confidence in ambiguous cases, prompting the user to provide clearer input, while a “content” state could reinforce trust during reliable predictions. Such affective transparency could enhance usability in applications like mental health monitoring, empathetic virtual agents, or driver safety systems, where understanding the AI’s operational state is as important as its output. Despite these promising results, several considerations warrant further exploration. The emotional features used in this study as the output of the I-Center were heuristically defined; future work could explore data-driven approaches to optimize the translation from calculation metrics to emotional dimensions and emotional features. Additionally, while short-term emotional states were modeled effectively, longer-term affective traits (e.g., a model’s “temperament” across training epochs) remain unexplored and could provide deeper insights into learning dynamics and model robustness. Ethical dimensions also merit attention: an AI capable of expressing emotional states must do so transparently and honestly, without manipulation or affective deception, particularly in sensitive domains like healthcare or education. In conclusion, this study provides the first evidence that emotional introspection, as implemented through the I-Center framework, can meaningfully enhance the performance and interpretability of facial emotion recognition systems. By endowing AI with a structured, emotionally grounded self-awareness, we move closer to models that are not only more accurate but also more transparent, communicative, and aligned with human social cognition. This work suggests that the future of affective computing lies not only in improving how AI recognizes human emotions, but also in advancing how AI understands and expresses its own internal states—a step toward more intuitive, and trustworthy human–AI partnerships. Declarations Author Contributions: K.S. did the analysis, wrote the main manuscript prepared figures and reviewed the manuscript Conflict of Interest: The authors declare no conflict of interest. Funding Declaration: SRG2023-00062-ICI, MYRG-GRG2024-00071-IC (University of Macau, China) Clinical trial number: not applicable. Consent to Publish declaration: not applicable. Consent to Participate declaration: not applicable. Data availability statement: The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request. Ethics statement: Ethics as not applicable. This study did not involve human research participants or live vertebrates. References Barredo Arrieta, A. et al. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58 , 82–115 (2020). Strelnikov, K. Internal Emotional Intelligence in AI Systems: An I-Center Framework for Human-Interpretable System States. Preprint at https://doi.org/10.20944/preprints202511.1142.v1 (2025). Russell, J. A. A circumplex model of affect. Journal of Personality and Social Psychology 39 , 1161–1178 (1980). Ko, B. A Brief Review of Facial Emotion Recognition Based on Visual Information. Sensors 18 , 401 (2018). Giannopoulos, P., Perikos, I. & Hatzilygeroudis, I. Deep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013. in Advances in Hybridization of Intelligent Methods: Models, Systems and Applications (eds Hatzilygeroudis, I. & Palade, V.) 1–16 (Springer International Publishing, Cham, 2018). doi:10.1007/978-3-319-66790-4_1. Goodfellow, I. J. et al. Challenges in representation learning: A report on three machine learning contests. Neural Networks 64 , 59–63 (2015). Li, S. & Deng, W. Deep Facial Expression Recognition: A Survey. IEEE Trans. Affective Comput. 13 , 1195–1215 (2022). Palmero, C. et al. Exploring Emotion Expression Recognition in Older Adults Interacting With a Virtual Coach. IEEE Trans. Affective Comput. 16 , 2303–2320 (2025). Poria, S., Cambria, E., Bajpai, R. & Hussain, A. A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion 37 , 98–125 (2017). Khare, S. K., Blanes-Vidal, V., Nadimi, E. S. & Acharya, U. R. Emotion recognition and artificial intelligence: A systematic review (2014–2023) and research recommendations. Information Fusion 102 , 102019 (2024). Zhang, R., Deng, H. & Xiao, X. The Insular Cortex: An Interface Between Sensation, Emotion and Cognition. Neurosci Bull 40 , 1763–1773 (2024). Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9047312","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":608398254,"identity":"8eb2034f-569a-4136-8100-d812ac9735d6","order_by":0,"name":"Kuzma Strelnikov","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA20lEQVRIie3PMQrCMBSA4SeF1yXQ9QlirxApFMGhV4kIdak3EBEK3XQuOHgFj1AIxsUDBHRQT1AQxEmMrkpbN4f88CDD++AFwGb7104FgEfvJ3Xq19GMMKSdQ2tuCGtOuH4TqCf+enE+ib2MgkO6u+hpn4Ert5sqwpUbcKGlEx7VME2UOYzFsa4kiEiilBjqpJcmaAixsJL4Gbp3Q1iQv8ijAQGFCOYw4mTIJGtAuIodEvsxJx0PV5MlMaz7i5+qVlmqQeTlo+Ka3GZdz5Wq+rCP8Ld1m81ms33rCcmcRjMEGnyaAAAAAElFTkSuQmCC","orcid":"","institution":"University of Macau","correspondingAuthor":true,"prefix":"","firstName":"Kuzma","middleName":"","lastName":"Strelnikov","suffix":""}],"badges":[],"createdAt":"2026-03-06 07:24:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9047312/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9047312/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":104998424,"identity":"0deedf01-3e13-4812-a49c-6be6a46096c9","added_by":"auto","created_at":"2026-03-19 16:26:35","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":161979,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePerformance improvement for the emotionally enhanced simple CNN model\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ea.\u003c/strong\u003e Bootstrap distribution of accuracy improvement (percentage points, pp) for the enhanced model over the standard CNN (n = 1000 resamples). Mean improvement = 8.14 pp; 95% confidence interval (CI) shown in green. Zero lies outside the CI, confirming statistically significant improvement.\u003cbr\u003e\n \u003cstrong\u003eb.\u003c/strong\u003e Statistical power as a function of sample size for detecting the enhancement effect. Dashed line indicates the conventional 80% power threshold.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ec.\u003c/strong\u003e Class‑wise accuracy improvement with 95% confidence intervals (horizontal error bars).\u003cbr\u003e\n \u003cstrong\u003ed.\u003c/strong\u003e Error analysis comparing standard and enhanced models. \u003cbr\u003e\nCNN, convolutional neural network; I-Center, introspective emotional feature generator.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-9047312/v1/38108d0e458222e90b1a71b6.png"},{"id":104998439,"identity":"b31017d6-6688-4a33-9dd7-ddde483b8d3f","added_by":"auto","created_at":"2026-03-19 16:26:44","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":163266,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePerformance improvement for the emotionally enhanced CNN-2 model\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ea.\u003c/strong\u003e Bootstrap distribution of accuracy improvement (percentage points, pp) for the enhanced CNN‑2 model over its standard counterpart (n = 1000 resamples). Mean improvement = 6.71 pp; 95% confidence interval (CI) excludes zero, confirming statistically significant enhancement.\u003cbr\u003e\n \u003cstrong\u003eb.\u003c/strong\u003e Statistical power as a function of sample size for detecting the improvement effect. \u003cbr\u003e\n \u003cstrong\u003ec.\u003c/strong\u003e Class‑wise accuracy improvement with 95% confidence intervals (horizontal error bars). \u003cbr\u003e\n \u003cstrong\u003ed.\u003c/strong\u003e Error analysis comparing standard and enhanced models. \u003cbr\u003e\nCNN‑2, second convolutional neural network architecture with increased channel dimensions; I‑Center, introspective emotional feature generator.\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-9047312/v1/6bdba792f3bc2ee68a0f93b3.png"},{"id":104998444,"identity":"93653c2f-f099-40aa-b33d-a3853e8b283a","added_by":"auto","created_at":"2026-03-19 16:26:46","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":167105,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePerformance improvement for the emotionally enhanced Resnet model\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ea.\u003c/strong\u003e Bootstrap distribution of accuracy improvement (percentage points, pp) for the enhanced ResNet‑18 model over the standard version (n = 1000 resamples). Mean improvement = 5.22 pp; 95% confidence interval (CI) excludes zero, confirming statistical significance.\u003cbr\u003e\n \u003cstrong\u003eb.\u003c/strong\u003e Statistical power as a function of sample size. \u003cbr\u003e\n \u003cstrong\u003ec.\u003c/strong\u003e Class‑wise accuracy improvement with 95% confidence intervals (horizontal error bars). \u003cbr\u003e\n \u003cstrong\u003ed.\u003c/strong\u003e Error analysis comparing standard and enhanced ResNet models.\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-9047312/v1/c4f4c97212eded702c33518c.png"},{"id":109281013,"identity":"99b5d0e3-7b2f-42d4-bc98-77122852bbd8","added_by":"auto","created_at":"2026-05-14 17:39:50","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":565297,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9047312/v1/66d38b3a-51fe-44ed-b006-01c7c286d305.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Artificial emotional introspection improves learning for facial emotion recognition","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe drive toward more transparent, communicative, and cognitively aligned artificial intelligence (AI) has given rise to an emerging focus on explainable AI to be able to monitor, interpret, and express their internal computational states in human-interpretable ways \u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e. Besides, while extensive research in affective computing has concentrated on enabling AI to recognize human emotions, the inverse capability - endowing AI with a structured form of emotional experience - remains unexplored. Recently, we proposed the I-Center framework as a computational model of artificial introspection \u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e, translating real-time performance metrics such as processing latency, confidence scores, and input anomalies into the classic psychological model of emotions \u003csup\u003e\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e - a dimensional emotional space defined by valence and arousal. This framework provides AI with an \u0026ldquo;inner observer\u0026rdquo; capable of articulating states like \u0026ldquo;content,\u0026rdquo; \u0026ldquo;stressed,\u0026rdquo; \u0026ldquo;focused\u0026rdquo; etc. based on its operational well-being, thereby externalizing the black-box nature of learning into an emotionally resonant narrative.\u003c/p\u003e \u003cp\u003eFacial emotion recognition (FER) presents a compelling and socially relevant domain \u003csup\u003e\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u003c/sup\u003e for implementing and validating this introspective approach. As a task that is both psychologically nuanced and computationally challenging, FER requires models to generalize across substantial variability in lighting, pose, expression intensity, and demographic attributes\u0026mdash;all while mapping visual patterns to categorical emotional states. The widely adopted FER-2013 dataset \u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e,\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e encapsulates these challenges, offering a realistic \u0026ldquo;in-the-wild\u0026rdquo; benchmark where human-level performance of 60\u0026ndash;70% remains a meaningful target for machine models \u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u003c/sup\u003e. Conventional deep learning approaches \u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e, though effective, often treat the training process as an optimization black box \u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e, driven solely by loss minimization without any integrative feedback about the model\u0026rsquo;s own \u0026ldquo;cognitive-emotional\u0026rdquo; state during learning. This limits not only interpretability but also the potential for the model to adaptively regulate its learning behavior in response to difficulty, ambiguity, or confidence.\u003c/p\u003e \u003cp\u003eIn this study, we investigate whether integrating the I-Center framework into the training pipeline of FER systems can enhance learning and generalization. We employ the I-Center to transform real-time training signals\u0026mdash;such as batch loss, gradient norms, inference time, and prediction confidence\u0026mdash;into a consistent multi-dimensional emotional feature vector that represents the network\u0026rsquo;s introspective state \u003csup\u003e\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u003c/sup\u003e. These emotional features are then fused with visual features extracted from facial images, providing the classifier with concurrent access to both perceptual content and emotional self-awareness throughout training.\u003c/p\u003e \u003cp\u003eOur objective is to empirically evaluate whether such introspective emotional feedback leads to measurable improvements in facial emotion recognition accuracy. We test this hypothesis across multiple neural architectures, from standard convolutional networks to attention-augmented ResNet models, training each in two configurations: a conventional baseline and an emotionally enhanced variant that receives I-Center-generated features. Through rigorous statistical evaluation, we assess whether emotional introspection yields significant gains in performance, and we discuss the implications of this approach for creating more transparent, adaptive, and psychologically plausible affective AI systems. This work thus bridges the paradigm of introspective AI, as instantiated by the I-Center, with the applied challenges of facial emotion recognition, offering a step toward models that not only decipher emotions but also \u0026ldquo;feel\u0026rdquo; and learn from their own internal states.\u003c/p\u003e"},{"header":"Materials and Methods","content":"\u003cp\u003e\u003cem\u003eFace dataset\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe Facial Expression Recognition 2013 (FER‑2013) dataset has human recognition performance estimated at approximately 65 \u0026plusmn; 5%, establishing a reference point for evaluating computational models\u0026nbsp;\u003csup\u003e5\u003c/sup\u003e. The challenging nature of FER‑2013 has made it a cornerstone for benchmarking facial emotion recognition systems. Introduced by Pierre Luc Carrier and Aaron Courville \u0026nbsp;in the ICML 2013 workshop\u0026rsquo;s facial expression recognition challenge\u0026nbsp;\u003csup\u003e6\u003c/sup\u003e, the dataset contains 35,887 facial images collected via the Google image search API using emotion‑related keywords, presenting a realistic\u0026nbsp;\u0026ldquo;in‑the‑wild\u0026rdquo;\u0026nbsp;setting with substantial variation in subject age, pose, lighting, and partial occlusion. The dataset is annotated with seven emotion categories: anger (4,953 images), disgust (547), fear (5,121), happiness (8,989), sadness (6,077), surprise (4,002), and neutral (6,198). This distribution reflects natural imbalances, with happiness being the most frequent and disgust the rarest class. The dataset is split into three subsets: 28,709 images for training, 3,589 for public validation, and 3,589 for final testing.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eMain workflow\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThie main workflow examines implementations of a facial emotion recognition system that integrates an I‑Center module\u0026nbsp;\u003csup\u003e2\u003c/sup\u003e\u0026mdash;a computational introspection mechanism that translates training metrics into emotional features. The implementations share a core philosophy: enhancing a neural network\u0026rsquo;s learning process by providing it with simulated emotional feedback.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe first implementation, built around a simple and more sophisticated CCNs (see below), establishes the main workflow. The system trains two models simultaneously: a standard CNN serves as a baseline, while an enhanced version receives supplementary input. After each training batch, the performance of the standard model\u0026mdash;including its loss, prediction confidence, and gradient magnitudes\u0026mdash;is analyzed by the I‑Center. The I-Center synthesizes a vector representing the calculation-related emotional state. The emotional feature vector is concatenated with the flattened visual features extracted by the CNN\u0026apos;s convolutional layers, forming a combined input to the enhanced classifier network. This means the emotional context is integrated at the beginning of the decision-making process, allowing all subsequent fully connected layers to process both visual and introspective information together.\u003c/p\u003e\n\u003cp\u003eIn the ResNet implementation, the workflow follows a similar sequence. A standard ResNet‑18 model first processes a batch of face images, and its performance metrics (actual inference time, loss, gradients, and confidence) are measured in real time. These raw signals are passed to the I‑Center, which transforms them into an emotional feature vector representing the model\u0026rsquo;s simulated introspective state. These emotional features are then fused with the visual features extracted by ResNet using a dedicated multi‑head attention module. This attention mechanism allows the visual features to dynamically query and weight the emotional context, creating an integrated representation that blends perceptual and introspective information. The fused features are finally passed through the classifier layers to produce the emotion prediction.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThroughout training, both the standard and enhanced (emotion‑aware) models are updated in parallel, and their comparative performance is assessed to quantify the benefit of emotional introspection.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eSimple convolutional neural network\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe architecture consists of three sequential convolutional blocks, each containing a convolutional layer, batch normalization, ReLU activation, max pooling, and dropout for regularization. Starting with 32 filters, the model progressively increases depth to 64 and then 128 channels while reducing spatial dimensions from 48\u0026times;48 down to 6\u0026times;6 feature maps. The flattened features are then passed through a two-layer fully connected classifier with 256 hidden units and a final output layer of 7 emotion classes.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eSecond convolutional neural network\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThis model introduces some architectural modifications/enhancements of the previously described simple CNN. The most significant modifications include increased channel dimensions across the first two convolutional blocks\u0026mdash;expanding from 32 to 40 filters in the first layer, from 64 to 80 in the second. Additionally, the hidden layer in the classifier is reduced from 256 to 216 units, potentially optimizing the parameter-to-performance ratio while preventing overfitting. The modifications are an example of different possible architecture options.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eHybrid convolutional neural network with attention and transfer learning\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe model adapts a pre-trained ResNet18 for grayscale facial images by modifying its initial layer, then enhances its feature extraction by inserting four attention units after each of its main building blocks. Each attention unit works in two complementary ways: first, it identifies which feature channels (like edges, textures, or shapes) are most important and boosts their influence; second, it pinpoints spatial regions within the image (such as eyes, mouth, or brow) that are most relevant to emotion. This dual focus allows the network to selectively amplify meaningful facial cues while ignoring irrelevant background noise, making it particularly adept at detecting subtle emotional expressions.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eI-Center\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe I-Center is a computational module designed to simulate a form of machine introspection by translating the raw metrics of a neural network\u0026apos;s training process into structured emotional states and feature vectors. It operates through three specialized subcenters: an Estimation Subcenter that converts technical measurements (inference time, loss values, gradient norms, and classification confidence) into psychological-like parameters like efficiency, stress, and confidence; an Emotional Subcenter that maps these parameters onto a continuous two‑dimensional emotional space (valence and arousal), producing human‑interpretable states such as \u0026quot;Content,\u0026quot; \u0026quot;Focused,\u0026quot; \u0026quot;Concerned,\u0026quot; or \u0026quot;Excited\u0026quot;; and a Generative Subcenter that synthesizes a consistent 15‑dimensional feature vector. These 15 generated features constitute a multi-dimensional emotional signature that integrates four key aspects of the model\u0026apos;s cognitive\u0026ndash;affective state: (1) emotion identity, indicating which facial expression category is currently being processed; (2) introspective state, reflecting internal metrics such as task difficulty, confidence, stress; (3) emotional state projection, derived from the valence\u0026ndash;arousal representation via a cosine component; and (4) process health, captured through a binary flag that indicates whether the current inference time is within the expected normal range. Together, these features enable a structured, interpretable representation of how the model\u0026rsquo;s learning dynamics align with human-like emotional progression during training.\u003c/p\u003e\n\u003cp\u003eTo ensure realistic and grounded behavior, the I‑Center first establishes a performance baseline using a processing time benchmark, which measures normal inference times across different emotion classes. This allows the system to detect anomalies in processing speed and to weight emotional responses according to the inherent difficulty of each emotion category\u0026mdash;for example, recognizing that \u0026quot;disgust\u0026quot; is computationally more challenging than \u0026quot;happiness.\u0026quot; By dynamically adjusting stress, comfort, and efficiency scores based on real‑time training progress, epoch, batch diversity, and historical trends, the I‑Center creates an introspective narrative, replacing low‑level optimization signals with human-like emotional feature representations.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eThe simple CNN model achieved a baseline accuracy of 52.86% on the FER2013 test set. When enhanced with emotional introspection from the I‑Center, accuracy increased to 60.99%\u0026mdash;an absolute improvement of 8.14 percentage points (a relative gain of 15.4%), Figure 1.\u003c/p\u003e\n\u003cp\u003eStatistical analysis confirmed this improvement is highly significant. A McNemar\u0026rsquo;s test, which compares the paired predictions of both models, yielded a p-value of effectively zero (p \u0026lt; 0.000001), providing strong evidence that the enhanced model performs differently from the baseline. Bootstrap analysis with 1,000 resamples produced a 95% confidence interval for the improvement of [7.05, 9.15] percentage points, confirming the gain is reliable and not due to chance.\u003c/p\u003e\n\u003cp\u003eFor the second CNN, the emotionally enhanced version achieved a 6.71 percentage point (12.62% relative) increase in accuracy over the baseline CNN. Statistical validation confirmed the robustness of these results: McNemar\u0026apos;s test (p = 0.000000), paired t-test (p = 0.000000), and bootstrap confidence intervals (95% CI: [5.67, 7.73] percentage points) all reject the null hypothesis, providing strong evidence that the enhancement yielded meaningful improvement, Figure 2.\u003c/p\u003e\n\u003cp\u003eThe ResNet-18 model, augmented with emotional introspection via the I‑Center, demonstrated a clear and statistically significant performance gain. The baseline ResNet-18 achieved a test accuracy of 62.16%. The enhanced version, which integrated emotional features through an attention‑based fusion mechanism, reached 67.39% accuracy\u0026mdash;representing an absolute improvement of 5.22 percentage points and a relative improvement of 8.40%, Figure 3.\u003c/p\u003e\n\u003cp\u003eThe improvement is statistically robust. A McNemar\u0026rsquo;s test produced a p‑value of effectively zero (p \u0026lt; 0.000001), strongly rejecting the hypothesis that the two models perform identically. Bootstrap analysis further confirmed the reliability of this gain, with a 95% confidence interval of [4.17%, 6.28%] for the true improvement, indicating the result is not due to random variation.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThe results of this study demonstrate a statistically significant performance improvement when integrating the I-Center emotional introspection framework into various facial emotion recognition architectures. Across both standard CNN models and a more sophisticated ResNet backbone, the emotionally enhanced variants consistently outperformed their standard counterparts, with improvements ranging from 5 to 8 percentage points. This pattern suggests that the benefit of emotional introspection is not architecture-specific but can be generalized to several types of neural models, This finding underscores the value of the I-Center approach as a promising enhancement mechanism. On the other hand, recognition accuracy improvements varied across facial emotion categories in different CNN architectures (see Figs.\u0026nbsp;1\u0026ndash;3). This variability suggests that the choice of CNN may be guided by which specific emotions are prioritized for a given application \u003csup\u003e\u003cspan additionalcitationids=\"CR8\" citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e .\u003c/p\u003e \u003cp\u003eAt its core, the I-Center addresses a critical gap in conventional training paradigms: the absence of a high-level, psychologically grounded feedback signal. Traditional training relies solely on low-level optimization metrics such as loss and accuracy, which lack intuitive interpretability for either the model or a human observer. The I-Center bridges this gap by transforming these opaque metrics into a more human-like emotional narrative\u0026mdash;translating computational \u0026ldquo;stress\u0026rdquo; during challenging samples or \u0026ldquo;confidence\u0026rdquo; during correct predictions into an emotional feature vector. This introspective feedback provides the model with a richer, more contextual understanding of its own learning process, effectively enabling a form of artificial metacognition. By integrating emotional context at the feature level, whether through simple concatenation in CNNs or attention-based fusion in ResNet, the model gains access to a continuous stream of self-awareness that appears to regularize learning and improve generalization, particularly on challenging, real-world datasets like FER-2013.\u003c/p\u003e \u003cp\u003eNotably, the I-Center\u0026rsquo;s design aligns with emerging neurocomputational perspectives on affective intelligence \u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e. Just as the human brain integrates visceral, sensory, and emotional data to form a cohesive sense of self and guide decision-making, the I-Center enables AI models to synthesize computational metrics into an artificial form of self-awareness. This integration mimics the function of the insular cortex in the human brain, which plays a crucial role in creating a subjective sense of the body's condition \u003csup\u003e\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u003c/sup\u003e. By incorporating this neurobiologically-inspired introspective capability, AI systems can potentially achieve a more nuanced, context-aware form of information processing that goes beyond mere pattern recognition. This form of artificial affect is grounded in performance rather than arbitrary simulations of emotions. The resulting affective loop may help the model navigate ambiguous samples, reweight attention toward emotionally salient features, or dynamically adjust learning sensitivity in response to training difficulty. This adjustment brings artificial neural networks closer to a human‑like cognitive architecture that integrates emotion with learning.\u003c/p\u003e \u003cp\u003eThe bidirectional potential of the I-Center also opens new pathways for emotionally aligned human-AI interaction. In a facial emotion recognition context, the I-Center could enable an AI not only to classify a user\u0026rsquo;s emotion but also to express its own confidence or uncertainty in that classification through emotionally resonant cues. For instance, an \u0026ldquo;anxious\u0026rdquo; system state could signal low confidence in ambiguous cases, prompting the user to provide clearer input, while a \u0026ldquo;content\u0026rdquo; state could reinforce trust during reliable predictions. Such affective transparency could enhance usability in applications like mental health monitoring, empathetic virtual agents, or driver safety systems, where understanding the AI\u0026rsquo;s operational state is as important as its output.\u003c/p\u003e \u003cp\u003eDespite these promising results, several considerations warrant further exploration. The emotional features used in this study as the output of the I-Center were heuristically defined; future work could explore data-driven approaches to optimize the translation from calculation metrics to emotional dimensions and emotional features. Additionally, while short-term emotional states were modeled effectively, longer-term affective traits (e.g., a model\u0026rsquo;s \u0026ldquo;temperament\u0026rdquo; across training epochs) remain unexplored and could provide deeper insights into learning dynamics and model robustness. Ethical dimensions also merit attention: an AI capable of expressing emotional states must do so transparently and honestly, without manipulation or affective deception, particularly in sensitive domains like healthcare or education.\u003c/p\u003e \u003cp\u003eIn conclusion, this study provides the first evidence that emotional introspection, as implemented through the I-Center framework, can meaningfully enhance the performance and interpretability of facial emotion recognition systems. By endowing AI with a structured, emotionally grounded self-awareness, we move closer to models that are not only more accurate but also more transparent, communicative, and aligned with human social cognition. This work suggests that the future of affective computing lies not only in improving how AI recognizes human emotions, but also in advancing how AI understands and expresses its own internal states\u0026mdash;a step toward more intuitive, and trustworthy human\u0026ndash;AI partnerships.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003eAuthor Contributions: K.S. did the analysis, wrote the main manuscript prepared figures and reviewed the manuscript\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eConflict of Interest: The authors declare no conflict of interest.\u003c/p\u003e\n\u003cp\u003eFunding Declaration: SRG2023-00062-ICI, MYRG-GRG2024-00071-IC (University of Macau, China)\u003c/p\u003e\n\u003cp\u003eClinical trial number: not applicable.\u003c/p\u003e\n\u003cp\u003eConsent to Publish declaration: not applicable.\u003c/p\u003e\n\u003cp\u003eConsent to Participate declaration: not applicable.\u003c/p\u003e\n\u003cp\u003eData availability statement: The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003eEthics statement: Ethics as not applicable. This study did not involve human research participants or live vertebrates.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eBarredo Arrieta, A. \u003cem\u003eet al.\u003c/em\u003e Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. \u003cem\u003eInformation Fusion\u003c/em\u003e \u003cstrong\u003e58\u003c/strong\u003e, 82\u0026ndash;115 (2020).\u003c/li\u003e\n\u003cli\u003eStrelnikov, K. Internal Emotional Intelligence in AI Systems: An I-Center Framework for Human-Interpretable System States. Preprint at https://doi.org/10.20944/preprints202511.1142.v1 (2025).\u003c/li\u003e\n\u003cli\u003eRussell, J. A. A circumplex model of affect. \u003cem\u003eJournal of Personality and Social Psychology\u003c/em\u003e \u003cstrong\u003e39\u003c/strong\u003e, 1161\u0026ndash;1178 (1980).\u003c/li\u003e\n\u003cli\u003eKo, B. A Brief Review of Facial Emotion Recognition Based on Visual Information. \u003cem\u003eSensors\u003c/em\u003e \u003cstrong\u003e18\u003c/strong\u003e, 401 (2018).\u003c/li\u003e\n\u003cli\u003eGiannopoulos, P., Perikos, I. \u0026amp; Hatzilygeroudis, I. Deep Learning Approaches for Facial Emotion Recognition: A Case Study on FER-2013. in \u003cem\u003eAdvances in Hybridization of Intelligent Methods: Models, Systems and Applications\u003c/em\u003e (eds Hatzilygeroudis, I. \u0026amp; Palade, V.) 1\u0026ndash;16 (Springer International Publishing, Cham, 2018). doi:10.1007/978-3-319-66790-4_1.\u003c/li\u003e\n\u003cli\u003eGoodfellow, I. J. \u003cem\u003eet al.\u003c/em\u003e Challenges in representation learning: A report on three machine learning contests. \u003cem\u003eNeural Networks\u003c/em\u003e \u003cstrong\u003e64\u003c/strong\u003e, 59\u0026ndash;63 (2015).\u003c/li\u003e\n\u003cli\u003eLi, S. \u0026amp; Deng, W. Deep Facial Expression Recognition: A Survey. \u003cem\u003eIEEE Trans. Affective Comput.\u003c/em\u003e \u003cstrong\u003e13\u003c/strong\u003e, 1195\u0026ndash;1215 (2022).\u003c/li\u003e\n\u003cli\u003ePalmero, C. \u003cem\u003eet al.\u003c/em\u003e Exploring Emotion Expression Recognition in Older Adults Interacting With a Virtual Coach. \u003cem\u003eIEEE Trans. Affective Comput.\u003c/em\u003e \u003cstrong\u003e16\u003c/strong\u003e, 2303\u0026ndash;2320 (2025).\u003c/li\u003e\n\u003cli\u003ePoria, S., Cambria, E., Bajpai, R. \u0026amp; Hussain, A. A review of affective computing: From unimodal analysis to multimodal fusion. \u003cem\u003eInformation Fusion\u003c/em\u003e \u003cstrong\u003e37\u003c/strong\u003e, 98\u0026ndash;125 (2017).\u003c/li\u003e\n\u003cli\u003eKhare, S. K., Blanes-Vidal, V., Nadimi, E. S. \u0026amp; Acharya, U. R. Emotion recognition and artificial intelligence: A systematic review (2014\u0026ndash;2023) and research recommendations. \u003cem\u003eInformation Fusion\u003c/em\u003e \u003cstrong\u003e102\u003c/strong\u003e, 102019 (2024).\u003c/li\u003e\n\u003cli\u003eZhang, R., Deng, H. \u0026amp; Xiao, X. The Insular Cortex: An Interface Between Sensation, Emotion and Cognition. \u003cem\u003eNeurosci Bull\u003c/em\u003e \u003cstrong\u003e40\u003c/strong\u003e, 1763\u0026ndash;1773 (2024). \u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-9047312/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9047312/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eWhile facial emotion recognition (FER) systems have advanced significantly, they remain constrained by conventional training paradigms that rely solely on low-level optimization signals, without any mechanism for the model to reflect on or adaptively respond to its own learning experience. Here, we integrate the I-Center, a computational framework for artificial introspection, into the training pipeline of FER models. The I‑Center translates real‑time training metrics such as loss, gradient flow, inference time, and prediction confidence into an emotional feature vector grounded in the psychological valence\u0026ndash;arousal emotional model, providing the network with a continuous introspective narrative of its own operational state in a psychologically defined format. Across several architectures, including standard convolutional neural networks and attention‑augmented ResNet, emotionally enhanced models significantly outperformed their baseline counterparts on the FER‑2013 dataset. These results show that psychologically grounded, emotionally informed introspection can enhance learning and generalization in FER, moving beyond AI that merely recognizes emotion toward systems that learn with an integrated awareness of their own cognitive emotional state. This work bridges machine introspection with affective computing, offering a pathway toward more transparent, adaptive, and psychologically plausible human‑AI interaction.\u003c/p\u003e","manuscriptTitle":"Artificial emotional introspection improves learning for facial emotion recognition","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-19 16:26:29","doi":"10.21203/rs.3.rs-9047312/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"1c9da536-c56b-4d51-b1cf-13b126e15e79","owner":[],"postedDate":"March 19th, 2026","published":true,"recentEditorialEvents":[{"type":"decision","content":"Rejected","date":"2026-05-14T17:32:01+00:00","index":"","fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-14T17:39:10+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-19 16:26:29","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9047312","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9047312","identity":"rs-9047312","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}