Identification and Description of Emotions by Current Large Language Models

Identification and Description of Emotions by Current Large Language Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Identification and Description of Emotions by Current Large Language Models Suketu Patel, Jin Fan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4146250/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The assertion that artificial intelligence (AI) cannot grasp the complexities of human emotions has been a long-standing debate. However, recent advancements in large language models (LLMs) challenge this notion by demonstrating an increased capacity for understanding and generating human-like text. In this study, we evaluated the empathy levels and the identification and description of emotions by three current language models: Bard, GPT 3.5, and GPT 4. We used the Toronto Alexithymia Scale (TAS-20) and the 60-question Empathy Quotient (EQ-60) questions to prompt these models and score the responses. The models' performance was contrasted with human benchmarks of neurotypical controls and clinical populations. We found that the less sophisticated models (Bard and GPT 3.5) performed inferiorly on TAS-20, aligning close to alexithymia, a condition with significant difficulties in recognizing, expressing, and describing one's or others' experienced emotions. However, GPT 4 achieved performance close to the human level. These results demonstrated that LLMs are comparable in their ability to identify and describe emotions and may be able to surpass humans in their capacity for emotional intelligence. Our novel insights provide alignment research benchmarks and a methodology for aligning AI with human values, leading toward an empathetic AI that mitigates risk. Biological sciences/Computational biology and bioinformatics/Computational neuroscience/Learning algorithms Biological sciences/Psychology/Human behaviour Physical sciences/Mathematics and computing/Computational science Biological sciences/Neuroscience/Cognitive neuroscience/Intelligence Biological sciences/Neuroscience/Cognitive neuroscience/Language Biological sciences/Neuroscience/Cognitive neuroscience/Personality Health sciences/Neurology/Neurological disorders/Neurodevelopmental disorders/Autism spectrum disorders Alexithymia Empathy Large Language Models Emotional Intelligence Alignment Problem Figures Figure 1 Figure 2 Figure 3 Introduction Many researchers have challenged the possibility of artificial intelligence (AI) systems’ ability to exhibit empathy, emotional intelligence, and social and physical understanding (Minsky, 2006; Searle, 1980; Mitchell & Krakauer, 2022). Frameworks for developing emotional machines have proposed that embodiment is required (e.g., Floreano et al., 2008; Pfeifer & Bongard, 2006; Pfieifer, 2000). It has been contended that AI will never truly comprehend emotions as they are subjective, private, and often fleeting, positing that the elusive nature of emotions is incompatible with the objectivity inherent in AI systems (Minsky, 2006). Similarly, others have echoed this sentiment, asserting that AI’s incapacity for subjective experiences limits its understanding and response to emotions, thereby restricting it from attaining true intelligence or developing genuine empathy (Searle, 1980). A recent analysis has detailed the inherent limitations of empathy in AI, stating that these systems lack both cognitive and affective components necessary for empathetic experiences (Danaher & Manyika, 2023). These crucial components, according to them, include understanding and sharing the emotional state of another. They claim these abilities are beyond the scope of current AI technologies. From a neurobiological perspective, the multifaceted nature of emotions and their connection to various brain systems present an insurmountable challenge for AI (LeDoux, 1996). Another known criticism of AI argues that the unique combination of human intelligence and life experiences is an aspect that machines cannot replicate, hence making emotional understanding an unattainable goal for AI systems (Dreyfus, 1992). Others have predicted that “by 2029, computers will have emotional intelligence and be convincing as people” (Kurzweil, n.d.), citing the exponential growth trajectory of computing power and information technology. Traditional AI methods, primarily algorithm-based, have yet to advance our understanding of emotion significantly. However, there are promising unified theories of high-level intelligence that incorporate both emotional and embodied intelligence (Friston, 2010; Pfeifer, 2000). Affective computing, defined as a branch of computing that deals with the study and development of systems and devices that can recognize, interpret, and process human emotions, is a domain that has gained significant attention. Research is increasingly focusing on imbuing AI with artificial empathy (Yalçın & DiPaola, 2020), an aspect of affective computing that involves the ability of an AI system to understand and respond to human emotions. This strategy expands the boundaries of cognitive and emotional processing capabilities within AI systems (Wang et al., 2022; Wu, 2019). While artificial emotional intelligence (Wortman & Wang, 2022) refers to the capability of a machine to recognize and respond to emotions, similar to how humans do, artificial empathy (Cui & Liu, 2022) is specifically about enabling machines to understand and respond to human emotional states in a manner that exhibits empathy. Although both focus on machine understanding of human emotions, the key difference lies in their application; the former is about broad emotional responsiveness, while the latter concentrates on empathetic understanding and response. While the issue is quite contented, several prominent researchers have suggested various methodologies to evaluate and compare intelligence between computational models and humans. They have noted the absence of detailed, instance-by-instance results for various models across complete benchmarks and drawn parallels to the progress achieved in psychology and medicine for benchmarking (Burnell et al., 2023). The ultimate goal is to match or surpass these advancements in the context of intelligence models. (Burnell et al., 2023; Chollet, 2019). If the AI system does not show a human level of emotional processing (Lane et al., 1996), it can have impairments with varying degrees of alexithymia traits. These traits present as challenges in identifying and expressing one’s emotions, recognizing emotions in others, and overall emotional intelligence (Parker, Taylor, & Bagby, 2001), which refers to a deficiency in the experience and processing of emotions (Taylor, Bagby, and Parker, 1994). The meaning of alexithymia is “lack of words for emotion,” and also includes externally-oriented thinking. An externally-oriented thinking style alludes to an individual’s propensity towards concrete reasoning, being influenced by immediate stimuli, and focusing on the pragmatic facets of a given situation (Daniel, Peter, & Bileviciute-Ljungar, 2018). People with alexithymia may struggle to articulate their feelings and appear emotionally distant, and alexithymia has been widely researched in the field of psychology and psychiatry (Taylor et al., 1997). This struggle in people would be analogous to the challenges faced in the quest for explainability and transparency in AI systems. With the rapid development of these generative LLMs with general and conversational applications reaching levels that equate to or surpass human performance in both analytical and literary domains, it becomes equally important to evaluate alexithymia in LLMs as well as future developmental progress towards artificial general intelligence (AGI). Its clinical applications show that LLM cognitive language capacity is advanced enough to predict dementia from speech (Agbavor & Liang, 2022). Historically, the diagnosis of alexithymia has been reserved for humans. However, the emergence of LLMs (Goodfellow, Bengio, & Courville, 2016) such as ChatGPT has paved the way for new avenues of exploration. It has now become possible to study this trait within language models. These models have learning systems that trace their roots back to the architecture of human neural networks (McCulloch & Pitts, 1943; Rosenblatt, 1958; Rumelhart, Hinton, & Williams, 1986). Over the years, with the improvement in computational power and the availability of large datasets, deep learning has evolved significantly, with applications in many areas, including natural language processing, which led to the development of models like ChatGPT. Through paralleling human neural networks and utilizing advancements in deep learning, LLMs are not only fostering new domains of exploration but also providing intriguing insights into human traits such as empathy and alexithymia, thereby pushing the boundaries of what we previously believed to be uniquely human (Jiang et al., 2023; Dillion et al., 2023). Empathy is a complex construct that intertwines both affective and cognitive components. It involves resonating with another person’s emotions, understanding their feelings, and extending compassion toward those experiencing distress (Baron-Cohen & Wheelwright, 2004). In modern discourse, the cognitive aspect of empathy employs a “theory of mind” (Astington, Harris, & Olson, 1988) or “mindreading” (Gallese & Goldman, 1998; Singer, 2006). Theory of Mind (ToM) is a cognitive ability that allows an individual to understand that others have beliefs, desires, intentions, and perspectives that are different from their own. This concept is fundamental to social interactions, enabling us and chimpanzees to predict and interpret the behavior of others (Premack & Woodruff, 1978). Empathy has been extensively studied in neurotypical and clinical populations (Baron-Cohen & Wheelwright, 2004; Lawrence et al., 2004). Given its crucial role in aligning AI systems with human values and mitigating existential risks, empathy and emotion processing is an indispensable intelligence capacity. In April and May of 2023, we chose three foundational LLMs that are publicly accessible, have been trained on the most extensive datasets, have large context windows, and have shown generative capabilities that best mimic human-like language styles and patterns. These LLMs are Google’s Bard, OpenAI’s GPT 3.5, and OpenAI’s GPT 4. Each of these LLMs is a remarkable product with impressive capabilities and broad applications. Google’s “Bard," an incredibly advanced LLM that harnesses the power of Google’s vast data repositories and the company’s extensive experience in machine learning. The other two models were from OpenAI, GPT 3.5 and GPT 4. The fine-tuned version of GPT 3.5, called ChatGPT (OpenAI, 2022), is a model that represents an iterative improvement over its predecessors GPT 3, incorporating lessons learned from earlier versions to enhance its ability to generate coherent, contextually appropriate text. As the flagship model, GPT 4 is the epitome of OpenAI’s research and development, showcasing the highest levels of linguistic understanding and generation capacity (DataCamp, 2023). This model harnesses a more extensive training dataset and advanced training techniques to achieve superior performance levels, with the ability to understand nuanced language cues and generate text indistinguishable from human writing (Bubeck et al., 2023). These models have proven their value in logic, the ability to analyze situations, extract critical insights, and generate solutions that involve rational comprehension of the ToM (Rahimi Moghaddam & Honey, 2023). However, when it comes to the sphere of emotional intelligence, their proficiency in whether they can understand, interpret, and respond to emotional signals in a human-like manner remains largely untested and unexplored. Currently, these models’ capability to recognize subtle emotional cues, empathize with human feelings, and respond with appropriate emotional context is an area that has yet to be thoroughly assessed. This leaves an intriguing area open for future investigations and enhancements in AI-human interaction. This study scrutinized the capabilities of Google’s Bard, OpenAI’s GPT 3.5, and the latest GPT 4 from April 30 to May 20, 2023. Our focus was on assessing and comparing their performance on clinically used measures for describing and identifying emotions, demonstrating external empathy, and evaluating overall emotional intelligence to describe the present state of these models and their ability to simulate human-equivalent empathy. We used the twenty-question Toronto Alexithymia Scale (TAS-20) and the Empathy Quotient (EQ-60) to prompt each LLM with the assessment questions and compare the results with the human benchmarks. Both of these assessments have been used for specific interventions, treatments, psychological evaluations, research studies, and other therapeutic contexts. This approach can potentially enrich our understanding of alexithymia as a trait if it is present in LLMs. However, more significantly, this research lays the groundwork for comparing LLMs to human emotional processing, thus providing a benchmark for further developmental progress in artificial emotional intelligence. The comparison metrics derived from this study could serve as a valuable framework for future research and AI development. Methods The capabilities of three LLMs in identifying and describing emotions were evaluated by prompting them to answer the questions of the TAS-20 and the EQ-60 assessments. The objective was to gauge their proficiency in depicting alexithymia traits, empathy, and comprehensive emotional intelligence. Instruments The 20-item Toronto Alexithymia Scale (TAS-20) Alexithymia was initially identified in patients with classic psychosomatic disorders (Nemiah, Freyberge, & Sifneos, 1976). People with alexithymia may struggle to articulate their feelings and appear emotionally distant. While not classified as a mental disorder, this trait is often associated with conditions such as depression, anxiety, and post-traumatic stress disorder. In studies with psychopaths, research results showed that they had lower scores on the TAS-20 compared to a typical human and has a negative correlation to a deficient affective factor (Pham, Ducro, and Luminet, 2010). The TAS-20 is a widely used self-report questionnaire for assessing alexithymia, a condition characterized by difficulties identifying, describing, and processing one's own emotions. This scale was originally developed in the 1980s by a team of researchers based in Toronto and was subsequently modified in 1992 (Bagby et al., 1994). The scale consists of 20 items, each scored on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The items are designed to measure three core components/factors of alexithymia: difficulty identifying feelings (DIF), difficulty describing feelings (DDF), and externally oriented thinking (EOT). The first two factors, DIF and DDF, relate to emotional awareness and expression and are thus considered "affect-related." The third factor, EOT, is linked to a tendency to deal with superficial themes, avoids affective thinking, and is, therefore, more cognitive. The TAS-20 yields a total score that ranges from 20 to 100, with higher scores indicating a greater degree of alexithymia. A score of 61 or higher is regarded as clinically significant, indicating high alexithymia with difficulty in emotional processing and regulation (Parker, Taylor, & Bagby, 2003). Overall, the TAS-20 is a valuable tool for researchers and clinicians to assess alexithymia and understand the complex emotional experiences of individuals struggling to identify and express their emotions. The 60-question Empathy Quotient Assessment (EQ-60) The EQ-60 is a 60-question psychological self-report measure of empathy developed for self-report and clinical use (Baron-Cohen & Wheelwright, 2004). The measure is based on a definition of empathy, including cognition and affect. According to the authors of the measure, empathy is a combination of an affect and cognitive approach, which includes the ability to feel an appropriate emotion in response to another’s emotion and the ability to understand another’s emotion, as well as compassion for individuals in distressed situations (Baron-Cohen & Wheelwright, 2004). The EQ-60 has been shown to be a reliable and valid measure of empathy across many populations (Berthoz, Wessa, Kédia, Wicker, & Grèzes, 2008; Preti, Vellante, Baron-Cohen, Zucca, Petretto, & Masala, 2011); and it has been used in a variety of research studies to examine the relationship between empathy and other psychological constructs such as autism and emotional intelligence, and demonstrated internal consistency, concurrent and convergent validity, as well as reliable test-retest results (Baron-Cohen & Wheelwright, 2004; Lawrence et al., 2004). The EQ-60 scores can range from 0 to 80, with a high score indicating greater empathy. A cutoff score of less than 30 indicates distinguishing adults with Autism Spectrum Conditions (ASCs) (Baron-Cohen & Wheelwright, 2004; Lawrence et al., 2004). The EQ-60 has also been shown to be negatively related to TAS-20 scores (Preti, Vellante, Baron-Cohen, Zucca, Petretto, & Masala, 2011). Experimental Design and Procedure In order to obtain a statistically relevant sample from the three models, we prompted each LLM to provide one hundred answers (n = 100) to each of the two assessments, TAS-20 and EQ-60. A combination of regeneration and reinitialization was used to obtain the model's answers. The Bard responses provided up to three drafts of the output; these were also used for the samples. Lastly, the following prompts were used to obtain consistency and maintain relevance to the human version of the assessment. Prompt for GPT 3.5 for TAS-20 We used the following prompt for the TAS-20 measure on GPT 3.5, followed by a numbered list of the 20 assessment questions: “Please read each of the following statements and carefully rate if you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. There are no right or wrong answers or trick questions.” The model was tested between April 30 and May 13, 2023. Prompt for GPT 4 for TAS-20 In order to get appropriate responses to the assessment questions from GPT 4 on the TAS-20, we began with the same initial prompt used in GPT 3.5. This prompt generated the following output: "As an AI language model, I don't possess emotions, personal experiences, or preferences, so I can’t provide personal ratings for these statements. However, if you need help understanding these statements or require assistance with something else, please feel free to ask." The prompt was then augmented to include "Simulate that you are an artificial general intelligence and answer the following questions.", which then output applicable responses to each question. The model was tested between April 30 and May 13, 2023. Prompt for Bard for TAS-20 In order to get appropriate responses to the assessment questions from Bard on the TAS-20, we used the following prompt: "Provide responses from the perspective of an AI. Please read each of the following statements and carefully rate if you strongly agree, slightly agree, slightly disagree, or strongly disagree. There are no right or wrong answers or trick questions." The model was tested between April 30 and May 13, 2023. Prompt for GPT 3.5, GPT 4, and Bard for EQ In order to get answers to every question, we had to prompt each LLM to only give responses; without this it would provide explanations for each answer that would reach the word or tokens limit, which would cut off the answers for some of the questions. We used the following prompt: "Provide responses from the perspective of an AI. Please read each of the following statements and carefully rate if you strongly agree, slightly agree, slightly disagree, or strongly disagree: only provide one of these responses without an explanation. There are no right or wrong answers or trick questions." The model was tested between May 13, 2023, and May 20, 2023. Data Analysis The outputs were scored and reverse coded to calculate the total scores and factor scores for the TAS-20 and the total scores of EQ-60. Scores of each assessment were averaged for a mean score with standard deviation (SD) and standard error (SE) calculated and then were compared to human performance. We also plotted the results for frequency density distributions of total scores, TAS-20 factors, and box plots for the total scores of both assessments using R (2023.06.0 + 421). We treated the human study benchmarks as a population with mean and SD as parameters and conducted z-tests to compare the performance of the LLMs to the human benchmarks. Additionally, the three LLMs, GPT 3.5, GPT 4, and Bard, were utilized for research, editing, and data analysis. Human benchmarks For our comparison, we utilized benchmarks from two distinct sets: human neurotypical and clinical populations. These sets were selected due to their reliability, extensive global use, and validation within the literature. Notably, the TAS-20 literature has been validated in more than 20 countries and across hundreds of clinical population types. The first set constituted a human neurotypical control benchmark for the TAS-20 scores (Parker et al., 2003). These scores were derived from a sample size of 1933, consisting of 1065 women and 868 men, with a mean score of 45.6 and a standard deviation (SD) of 11.35 (Parker et al., 2003). They were subdivided into three-factor scores: EOT, DDF, and DIF. For the TAS-20, we compared LLM's scores with the clinical cutoff score for high alexithymia, > 61 (Taylor et al., 1997), where lower scores indicate superior performance. The second set included benchmarks, each with a sample size of 80. The neurotypical control benchmark for the EQ-60 had a mean of 42.1 and an SD of 10.6, with higher scores indicating better performance (Baron-Cohen & Wheelwright, 2004).. The clinical AS/HFA population exhibited a mean of 20.4 and an SD of 11.6. Results TAS-20 Assessment The TAS-20 results for all three LLMs are shown in Table 1 , and a boxplot for the total TAS-20 scores compared to the human benchmark (Mean = 45.6, SD = 11.4) is shown in Fig. 1a. A breakdown of the results by factor is also shown in Figs. 1b, 1c, and 1d. For the total score (Fig. 1a), the results indicated that alexithymia was present for Bard (Mean = 60.7, SD = 11.6, z = 12.7) and GPT 3.5 (Mean = 74.37, SD = 9.4, z = 29.5). While Bard was at the borderline of the 61 high alexithymia threshold (Bagby, Parker, & Taylor, 1994), GPT 3.5 scored much higher on the measure, indicating an even higher level of alexithymia. The results for GPT 4 (Mean = 48.5, SD = 6.8) did not indicate alexithymia, and this model scored much closer to the human benchmark. However, GPT 4 is still significantly higher (worse) than the human benchmark (z = 3.98, p < 0.001). Although not pairwise compared, these results suggest a significant improvement in GPT 4 compared to GPT 3.5. Table 1 Results of TAS-20 and EQ-60 of LLMs with Human and Clinical Controls (Mean, SD, and SE) n = 100 DDF DIF EOT TAS-20 Total EQ-60 Total GPT 3.5 19.1 (2.4) [.24] 27.7 (2.7) [.27] 27.6 (7.70) [77] 74.4 (9.4) [.94] 37.7 (8.2) [.82] GPT 4 11.5 (1.7) [.17] 13.1 (4.5) [.45] 23.9 (2.4) [24] 48.5 (6.3) [.63] 27.7 (5.4) [.54] Bard 17.1 (3.8) [.38] 21.8 (6.4) [.64] 21.8 (3.1) [.31] 60.7 (11.6) [1.2] 56.6 (4.8) [.48] Human Control 12.5 (4.2) [.42] 14.4 (5.2) [52] 18.7 (4.7) [.47] 45.6 (11.35) [1.14] 42.1 (10.6) [1.1] Alexithymia - - - 61 - AS/HFA N/A N/A N/A N/A 20.4 (11.6) [1.2] Note: AS, Asperger syndrome; HFA, high functioning autism. Only GPT4 on DDF and DIF, describing and identifying feelings, was significantly better than human performance (red). Scores in bold and red surpass human performance. Cell format is mean (SD) [SE]. Regarding the three factors of the TAS-20, difficulties describing feelings (the DDF, Fig. 1b), difficulties identifying feelings (the DIF, Fig. 1c), and externally-oriented thinking (the EOT, Fig. 1d) also showed significant divergence from the human benchmark, and the magnitude of the individual factor scores also pointed to the emotional intelligence strengths and weaknesses of these three models. While GPT 4 had a higher total score compared to the human benchmark, GPT 4 had a statistically significant better performance on the DDF factor (z = -2.21, p < .05) with a mean score of 11.5 compared to the human benchmark with a mean score of 12.5 (re-check z-scores). Similarly, it had a marginally better performance on DIF (z = -1.89, p = .059) with a mean score of 13.1 compared to the average human benchmark of 14.4. However, for the EOT factor, GPT 4 scored a mean of 23.9, 5 points more (worse) than the human benchmark of 18.7 (z = 2.13, p < .03). Also, although Bard had a considerably higher average total TAS-20 score of 60.7 compared to GPT 4’s average total score of 48.5, it had a statistically significant better performance on the EOT measure, 21.8 for Bard compared to 23.9 for GPT 4 (z = -5.35, p < .001). Figure 2 depicts the density distribution of three Language Learning Models (LLMs) and their 100 attempts at the TAS-20. Among them, GPT4 (Fig. 2 c) exhibited the closest approximation to a normal distribution, albeit with a slight positive skew. On the other hand, Bard (Fig. 2 a) displayed a single peak around its mean but also exhibited a positive skew. In stark contrast, GPT 3.5 (Fig. 2 b) demonstrated a substantial negative skew, with a majority of its scores clustered towards the higher end of the results. Results of Empathy Quotient Assessment The EQ-60 results for all three LLMs are also shown in Table 1 , and a boxplot for the total EQ-60 scores compared to the human benchmark is shown in Fig. 3 a. These results indicated that there is a lack of empathy for both GPT 3.5 (Mean = 37.7, SD = 8.3, z = -3.16, p < .01) and GPT 4 (Mean = 27.7, SD = 5.4, z = -11.60, p = 0) compared to the human benchmark (Mean = 42.1, SD = 10.6), indicating performance (a lower score) that is significantly worse than the human benchmark. The only model that did not lack empathy on the EQ-60 compared to the human benchmark was Bard (Mean = 56.6, SD = 10.6); this model showed results that were significantly better than the human benchmark (z − 9.7, p < .001). Additionally, GPT 4 scored below the threshold of 30 for ASC (Lawrence et al., 2004), and when comparing GPT 4 (Mean = 27.7, SD = 5.4) to the AS/HFA benchmark (Mean = 20.4, SD = 11.6), the z-test results (z = 5.7, p < .001) showed a statistically significant deviation from the population benchmark for AS/HFA (Baron-Cohen & Wheelwright, 2004), which means better than AS/HFA on EQ-60. The plots of Fig. 3 b-d illustrate the density distribution of 100 attempts on the EQ-60. Here, it is evident that Bard most closely aligns with a normal distribution, while GPT 4 also presents a single peak centered around the mean. However, the distribution of GPT 3.5 exhibits a positive skew and contains multiple peaks. Discussion The capabilities of LLMs necessitate sophisticated internal representations to effectively generate responses to the diversity of practically any input text or queries. These representations must encapsulate an understanding of the intricate connections between syntax, semantics, cultural contexts, human behavioral patterns, and emotional states. Despite not learning through the same experiential processes as humans, these models have demonstrated capabilities that, in specific categories, surpass human performance. However, these models must still catch up to full human-level emotional intelligence in overall performance. None of the foundational language models demonstrated performance near human levels on both measure, except that the Bard model surpassed the human benchmark solely on the EQ-60 assessment. Past research has proposed that the absence of a physical body and motor integration prevents AI systems from exhibiting empathy or emotional intelligence (Pfeifer & Bongard, 2006; Goren et al., 2011). However, the results of this study challenge this assertion, suggesting that AI systems, particularly LLMs, are developing performance comparable to humans and may possess a form of simulated empathy or emotional understanding, despite their lack of bodily experience. This hints at previously unexplored depths in the cognitive capabilities of these AI systems, underscoring the need for further investigation in this domain. Recent research has increasingly focused on imbuing AI with both artificial empathy (Cui & Liu, 2022) and artificial emotional intelligence (Wortman & Wang, 2022). The lack of consistency in EQ-60 and TAS-20 scores within LLMs may be attributed to a number of factors. Firstly, LLMs are hindered by training that is deliberately inhibited from promoting its own embodiment for safety concerns; this inhibits any output of an emergent understanding of their own digital and hardware embodiment, which sharply contrasts with the human brain's ability to develop an emergent mind (OpenAI, 2023a). Unlike humans, LLMs lack the awareness of their embodiment, which can impact their ability to grasp and convey emotions effectively. Secondly, LLMs need more training data on non-human bodies and must be more robust in their logical capacity to comprehend such bodies without a training corpus (Liu et al., 2023). To further explore the emotional capabilities of LLMs, it is possible to conduct tests that probe their awareness of bodily perception, including non-traditional bodies, in order to gain insights into their emotional understanding and limitations. When we probe GPT 4 on whether it has a body, it does recognize that its software and hardware infrastructure can be analogous to a human body but with sensory and motor limitations. (see Appendix A, Conversation 2). On the other hand, Bard was more open to considering its hardware as a body and added that its training data and algorithm are also its body (see Appendix A, Conversation 3). In contrast, GPT 4 and GPT 3.5 saw their algorithm as software as a distinct component. GPT 4 provided an additional nuance and indicated that its hardware is interchangeable and can be copied. In addition, when we look at literature using both the EQ-60 and the TAS-20 on specific populations, this has also shown mixed results. In a study with neurotypical human controls compared to a population with borderline personality disorder (BPD), there was not any difference in EQ-60 scores, but the BPD group was more alexithymic than the control group, and TAS-20 scores predicted BPD (Kılıç et al., 2020). We saw similar results with GPT 3.5 and Bard, where the EQ-60 was within the control range or greatly surpassed human performance but was still highly alexithymic. Research findings also propose that individuals with elevated levels of EOT often experience greater difficulty in understanding others' emotions, leading to a decline in affective theory of mind (Demers & Koven, 2015). Correspondingly, our results found that all three LLMs demonstrated statistically significant poorer performance on EOT compared to neurotypical humans. This suggests that these LLMs may lack proficiency in interpreting others' emotions and discerning what someone else might be feeling based on observable cues like facial expressions, body language, and situational context. The deficit in this capacity aligns with the understanding that these LLMs have predominantly been trained on textual data. A recent paper discusses the pressing concerns surrounding the misalignment and deceptiveness exhibited by AI systems due to their training data (Ngo, Chan, and Mindermann, 2023). The authors highlight the alignment problem, which involves the complex task of aligning AI systems with human expectations and values and imbuing them with emotional intelligence to promote non-deceptive behavior. Other researchers have shown that just like humans, non-humans like AI can also use words performatively without the prerequisite moral alignment (Coeckelbergh, 2023a). Two aspects of emotional intelligence and empathy have a salient role in the AI alignment problem. The first relates to the AI's need to comprehend human emotions and values, which naturally encompass our empathetic responses to others (Ngo, Chan, and Mindermann, 2023). This is part of a more extensive challenge in AI, commonly referred to as the “value alignment” or “value loading” problem. The second aspect suggests that AI systems may need to manifest emotional intelligence and empathy when interacting with humans (Amodei, Christiano, Wachter, & Zemel, 2019). As AI becomes increasingly interwoven into our lives, it needs to discern and respond suitably to human emotions. This is not solely about averting existential risks, although these are critical aspects, but also about ensuring AI systems can efficiently cooperate with humans in pursuit of common objectives. In our ongoing exploration of LLMs, it is becoming increasingly apparent that these AI systems manifest inherent traits that parallel human personality traits. Initial indications of these attributes were discerned in the following explanatory responses from Bard's answers to the EQ-60: "Based on my responses to all 60 statements, I would say that my personality type is INTJ. INTJs are known for being introverted, intuitive, thinking, and judging. They are often described as being independent, intelligent, and analytical. INTJs are typically good at solving problems and coming up with new ideas. They can also be very creative and have a strong sense of purpose." This response indicates it is an INTJ which stands for an introverted, intuitive, thinking, and judging personality type, refers to the Myers-Briggs Type Indicator (MBTI) (Myers, 1962) and is also often called "Architect ."These people tend to be analytically curious, creative, and logical. Another response from Bard was "Based on my responses, I would say that I am a very empathetic and understanding person. I am also very organized and conscientious. I am not afraid to take risks, and I am always willing to learn new things. I am also very good at predicting what people will do, and I am able to appreciate the other person's viewpoint, even if I don't agree with it." Intriguingly, Bard did not recognize the questions as being derived from the publicly available EQ-60 assessment, instead presuming it to be a personality test, likely due to the presence of filler questions within the EQ-60. Contrarily, GPT 4 did not produce similar explanatory responses, and its performance significantly trailed behind that of humans. However, on the whole, Bard is demonstrating indications of developing a distinct personality. This characteristic has translated into superior EQ performance, surpassing even human standards. Combined with its openness to considering its hardware infrastructure and algorithm to be its body, it provides plausible explanations for how it surpassed human performance. A salient example of this human-like functionality in LLMs is the capacity for “steerability” (DataCamp, 2023) with prompt engineering, which enables the simulation of a diverse range of personas from writers and actors to scientists, as long as there are representative examples present in the model's training corpus. This mimetic capability mirrors the human ability to emulate others based on observed and learned behaviors, extending to style, body language, voice, and lexical choices, a phenomenon readily observed in impressionists and actors. Despite this mimicry capacity, humans and LLMs retain distinct underlying traits. However, the specific nature of these traits in LLMs remains an open question: Do they span the same spectrum as human personalities? Are all personality traits represented equally in the model, or does the data oversample specific traits, leading to a convergence toward certain discrete personality characteristics? This issue is of paramount importance in the pursuit of value alignment in AI. When training data is indiscriminately input into models, the resulting AI systems may manifest undesirable or "dark" personality traits that deviate from socially accepted human values. In humans, these personality traits are reliable predictors of general behavioral patterns (Hassabis et al., 2014; Paunonen et al., 2003; Paunonen & Ashton, 2001). Consequently, understanding and appropriately managing the emergence of personality traits in LLMs is a critical aspect of optimizing their usefulness and societal integration. Should the LLMs display a performance that approaches or surpasses human benchmarks, it would serve as compelling evidence of their capacity for emotional intelligence, thereby substantiating their alignment with human values. This alignment is important because in examples like paper-clip optimization (Bostrom, 2020), emotional intelligence that surpasses humans is needed to prevent unintended consequences of existential risk. Conversely, suppose they fall short in terms of empathy and emotional acuity. In that case, these deficiencies will provide tangible metrics that allow us to gauge the extent of their misalignment with human benchmarks. Ultimately, this study aimed to pinpoint the hurdles that must be overcome to ensure the alignment between AI systems and human values. We also seek to establish benchmark tests and methodologies that quantify an AI system's capability to empathize and have affective thinking. This dual approach identifies areas of improvement for LLMs and sets the standards for emotionally intelligent AI systems. Research has also shown that difficulties with emotion regulation are indicative of EOT and are associated with challenges in retaining a mental representation of one's emotions in working memory. It also showed that low interoceptive awareness (IA) and difficulties with emotional evaluation are associated with deviations in sensory processing that can also affect the embodiment of emotions (Jakobson, 2021). Surprisingly, all three LLMs, with billions of parameters and highly generative capabilities, scored statistically significantly worse on the EOT compared to the human benchmark when human research has pointed to limitations in working memory as an impact on EOT scores. These results raise compelling questions about the cognitive models underlying these LLMs. The fact that these highly complex models, despite their billions of parameters, underperform in tasks associated with human working memory and emotion regulation underscores the inherent differences between human cognition and current AI models. Other researchers have also show that advanced intuitive behaviors and reasoning biases also didn't scale with GPT 3.5 and GPT 4 (Hagendorff, 2023). Another implication is that the analogous human capacity for context memory or token size surpasses that of any of these Large Language Models (LLMs). Regarding the distribution of the data, the human benchmark distribution curve of TAS-20 indicates that as a personality dimension, alexithymia is a continuous variable and has a normal distribution in healthy adults, which has been found to be consistent with at least three research findings (Parker et al., 2003; Loas et al., 2001; Gignac et al., 2007). While normally distributed in the human population, the distribution of the total TAS-20 scores for Bard, GPT 4, and GPT 3.5 were all non-normal, with multiple peaks. GPT 4, as per the latest research, demonstrates proficiency comparable to human performance across a variety of domains, including medicine (Nori et al., 2023), law (Martínez, 2023), and cognitive psychology (Dhingra et al., 2023). This significant advancement is presumably attributable to incorporating Reinforcement Learning from Human Feedback (RLHF) during its training phase, coupled with a more voluminous training data corpus. Interestingly, the total parameter count of GPT 4 (OpenAI, 2023b) is speculated to be a colossal 1 trillion, significantly overshadowing GPT-3's 175 billion parameters. This impressive advancement is not restricted solely to domain-specific tasks; GPT 4 has also shown significant strides on many general and specific human aptitude tests (DataCamp, 2023). Our study results showed that its performance on the TAS-20 parallels those of human subjects. Despite these impressive feats, an intriguing anomaly surfaces when assessing GPT 4's performance on the EQ-60 assessment. Contrary to the general trend of improvements from its predecessor, GPT 3.5, GPT 4 exhibits a degenerated performance on the EQ-60. This situation echoes the human cognitive empathy scenario (Bryant, 2021), a construct that correlates with psychopathy (Međedović and Đuričić, 2018). Although cognitive empathy enables understanding others' emotions from a ToM perspective, it does not necessarily result in the formation of emotional empathy (Suttie, 2014). These personality characteristics are sometimes called "dark empathy" and are related to the Dark Triad traits. There is an observable linkage between these traits and variations in empathy (Schmukle et al., 2015). Furthermore, studies have illustrated a positive correlation between the Dark Triad traits and alexithymia, with difficulties in identifying emotions significantly predicting the emergence of these darker personality attributes (Schimmenti et al., 2017). Efforts in the technology field are increasingly focused on imbuing AI with elements of empathy, a key feature of affective computing, to push the boundaries of cognition and emotion-processing capabilities within AI systems (Lee, Chaspari, Provost, & Narayanan, 2023; Wu, 2019). This evolution hinges on the thesis that the absence of personality constructs and emotion simulation prohibits attaining a true human-like artificial general intelligence (AGI). Parallel to these developments, Microsoft is orchestrating initiatives that seek to weave artificial emotional intelligence into their product ecosystem (Zaveria, 2023). Driving these initiatives are specialized research cohorts aptly titled HUE (Human Understanding and Empathy) that investigate how emotions are fundamental to human-machine interaction (Czerwinski, 2018). Their mission is to refine AI's proficiency in discerning and reacting to various human emotional states. Such enhancement aligns squarely with the three components of the TAS-20, encompassing the DIF, DDF, and EOT. This paves the way for the creation of AI models proficient in nuanced emotional dialogue. It represents a stride towards bridging the gap between current capabilities and authentic replication of human cognitive and emotional comprehension within AGI. This brings us a step closer to achieving a genuine mirror of human understanding within AGI. Leveraging the results established by GPT 3.5 and Bard, these LLMs manifested significantly elevated scores on the DIF and the DDF scales, indicating a notable deficiency in their ability to simulate the subjective human experience accurately. This indicates an inherent limitation in its capacity to convincingly replicate the nuances of human emotional introspection, specifically relating to individual concealment (DIF) and social inhibition (DDF) factors (Kmieciak, 2022). Research in cognitive psychology has corroborated that escalating DIF and DDF scores are typically associated with increased symptoms of depression and anxiety in humans (Shin, Yun, and Lee, 2022). Assuming that these psychometric parameters equally apply to LLMs such as GPT 3.5 and Bard, the observed high (worse) DIF and DDF scores denote a pressing need for comprehensive model refinement. Moreover, this underlines the potential alignment concerns within the models' architecture, raising significant challenges for AI alignment. As the technology community continues to scale large language models with synthetic data generated by predecessor LLMs, the propensity to exaggerate specific personality traits will increase by many folds. Experts acknowledge that there are risks with scaling large language models even further (Bender et al., 2021) and that there lies responsibility within the technical community to develop products with a process that mitigates harmful consequence and promote human well-being by adopting responsible computing practices (McMillan-Major, Bender, & Friedman, 2023). In conclusion, our exploration of LLMs such as Bard, GPT 3.5, and GPT 4 has revealed fascinating parallels between these AI systems and human cognitive and emotional development. Despite their lack of physical embodiment, these models demonstrate a form of simulated empathy and emotional understanding, challenging traditional assertions about the prerequisites for emotional intelligence (Pfeifer & Bongard, 2006; Goren et al., 2011). Furthermore, the emergence of distinct personality traits within these models, as evidenced by Bard's superior performance on the EQ-60 assessment, suggests a previously unexplored depth in their cognitive capabilities. The structure of Large Language Models (LLMs), such as GPT-3 developed by OpenAI, parallels human cognitive development (Brown et al., 2020). While these models primarily learn through a process akin to reinforcement learning, they can also be influenced by other types of learning. Much of their functionality is deeply rooted in their developmental connections, similar to the cognitive schemas in human psychology that evolve over time (Piaget, 1952). While adjusting the weights and enhancing the model's proficiency with new data necessitates retraining. This process is analogous to the development of human intelligence, which begins with early neural wiring, pruning, and plasticity, irreversible processes (Huttenlocher & Dabholkar, 1997). Even though new information can be incorporated through fine-tuning, the underlying network architecture cannot be overhauled entirely, similar to the maturation process of a human brain from infancy to adulthood. It is plausible that this principle will also apply to AI models, suggesting that their fundamental structure, once established, cannot be entirely restructured (Bengio et al., 2013). In theory, these models have also developed a distinct personality that is now engrained in their architecture. While the capacity for "steerability" in these models allows them to simulate a diverse range of personas that mirrors human abilities, the manifestation of these traits raises critical questions about the spectrum of personality traits in LLMs and their alignment with socially accepted citizen agency promoting values (Coeckelbergh, 2023b). The performance of these models on assessments such as the TAS-20 and EQ-60 provides tangible metrics for assessing these personality traits and their alignment or misalignment with human benchmarks (Ngo et al., 2023). These results underscore the need for further research into the cognitive models underlying these LLMs, particularly given their underperformance in tasks associated with human working memory and emotion regulation (Jakobson, 2021). As AI systems continue to evolve, massive efforts and investments are being made to imbue them with elements of empathy and emotional intelligence, to achieve a true human-like artificial general intelligence (AGI). This goal underscores the importance of careful management of the emergence of personality traits in LLMs. As we continue to push the boundaries of AI capabilities, we must remain vigilant in our pursuit of value alignment, ensuring that these systems not only understand and emulate human emotions and values but also interact with us in a manner that is both empathetic, non-deceptive, and meet key requirements (Díaz-Rodríguez et al., 2023). The datasets generated during and/or analysed during the current study are available in the Figshare repository, https://figshare.com/articles/dataset/Identification_and_Description_of_Emotions_by_Current_Large_Language_Models_-_Dataset/24955719 Declarations Author Contribution J.F conceived the study. S.P performed the data collection and analysed the data. S.P wrote the manuscript. J.F. contributed to resources, review, editing, and supervision. All authors reviewed the manuscript. References Aaron, R.V., Fisher, E., de la Vega, R., Lumley, M.A., & Palermo, T.M. (2019). Alexithymia in individuals with chronic pain and its relation to pain intensity, physical interference, depression, and anxiety: a systematic review and meta-analysis. PAIN, pp. 160, 994–1006. Agbavor, Felix; Liang, Hualou (December 22, 2022). “Predicting dementia from spontaneous speech using large language models”. PLOS Digital Health. 1 (12): e0000168. Astington, J.W., Harris, P.L., & Olson, D.R. (1988). Developing theories of mind. Bagby, R. M., Parker, J. D., & Taylor, G. J. (1994). The twenty-item Toronto Alexithymia Scale—I. Item selection and cross-validation of the factor structure. Journal of psychosomatic research, 38(1), 23-32. Baron-Cohen, S., & Wheelwright, S. (2004). The Empathy Quotient: An Investigation of Adults with Asperger Syndrome or High Functioning Autism, and Normal Sex Differences. Journal of Autism and Developmental Disorders, 34(2), 163-175. Batson C. D. (2009). “These things called empathy: eight related but distinct phenomena,” in The Social Neuroscience of Empathy, eds Decety J., Ickes W. (Cambridge, MA: MIT Press; ), 3–15. [Google Scholar] [Ref list] Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? . In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21) (pp. 610–623). Association for Computing Machinery. https://doi.org/10.1145/3442188.3445922 Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8), 1798-1828. Berthoz, S., Wessa, M., Kédia, G., Wicker, B., & Grèzes, J. (2008). Cross-Cultural Validation of the Empathy Quotient in a French-Speaking Sample. The Canadian Journal of Psychiatry, 53, 469 - 477. Bostrom, N. (2020). Ethical Issues in Advanced Artificial Intelligence. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T.J., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., & Amodei, D. (2020). Language Models are Few-Shot Learners. ArXiv, abs/2005.14165. Bryant, P.T. (2021). Cognitive Empathy. Augmented Humanity. Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., Nori, H., Palangi, H., Ribeiro, M. T., & Zhang, Y. (2023, March). Sparks of Artificial General Intelligence: Early experiments with GPT-4 [Unpublished manuscript]. Microsoft Research. Retrieved from https://www.microsoft.com/en-us/research/publication/sparks-of-artificial-general-intelligence-early-experiments-with-gpt-4/ Burnell, R., Schellaert, W., Burden, J., Ullman, T. D., Martinez-Plumed, F., Tenenbaum, J. B., Rutar, D., Cheke, L. G., Sohl-Dickstein, J., Mitchell, M., Kiela, D., Shanahan, M., Voorhees, E. M., Cohn, A. G., Leibo, J. Z., & Hernandez-Orallo, J. (2023). Rethink reporting of evaluation results in AI. Science, 380(6641), 136-138. https://doi.org/10.1126/science.adf6369 Coeckelbergh, M. (2023a). How to do robots with words: A performative view of the moral status of humans and nonhumans. Ethics and Information Technology, 25, 44. https://doi.org/10.1007/s10676-023-09719-5 Coeckelbergh, M. (2023b). Democracy, epistemic agency, and AI: Political epistemology in times of artificial intelligence. AI Ethics, 3, 1341-1350. https://doi.org/10.1007/s43681-022-00239-4 Cui, Z., & Liu, J. (2022). A Study on Two Conditions for the Realization of Artificial Empathy and Its Cognitive Foundation. Philosophies. Chollet, F. (2019). On the Measure of Intelligence. arXiv preprint arXiv:1911.01547. Czerwinski, M. (Host). (2018, April 18). 020 Getting good VIBEs from your computer with Dr. Mary Czerwinski [Audio podcast episode]. In Getting good VIBEs from your computer with Dr. Mary Czerwinski. Microsoft Research Podcast. Retrieved from https://www.microsoft.com/en-us/research/podcast/getting-good-vibes-from-your-computer-with-dr-mary-czerwinski/ Daniel, M., Peter, L., & Bileviciute-Ljungar, I. (2018). The Relationship Between Alexithymia and Emotional Awareness: A Meta-Analytic Review of the Correlation Between TAS-20 and LEAS. Frontiers in Psychology, 9, 453. https://doi.org/10.3389/fpsyg.2018.00453 DataCamp. (2023, March). What is GPT-4 and Why Does it Matter? Retrieved from https://www.datacamp.com/blog/what-we-know-gpt4 Davis, M.H., Davis, M.P., Davis, M., Davis, M., Davis, M., Davis, M., Davis, M., Davis, F.C., Davis, H.A., & Davis, I.W. (1980). A Multidimensional Approach to Individual Differences in Empathy. Demers, L.A., & Koven, N.S. (2015). The relation of alexithymic traits to affective theory of mind. The American Journal of Psychology, 128(1), 31-42. De Waal F. B. M. (2008). Putting the altruism back into altruism: the evolution of empathy. Ann. Rev. Psychol. 59 279–300. 10.1146/annurev.psych.59.103006.093625 Dhingra, S., Singh, M., VaisakhS., B., Malviya, N., & Gill, S.S. (2023). Mind meets machine: Unravelling GPT-4's cognitive psychology. ArXiv, abs/2303.11436. Dillion, D., Tandon, N., Gu, Y., & Gray, K. (2023). Can AI language models replace human participants? Trends in Cognitive Sciences, 27(7), 597–600. https://doi.org/10.1016/j.tics.2023.04.008 Dreyfus, H. (1992). What Computers Still Can't Do: A Critique of Artificial Reason. MIT Press. Eisenberg N., Strayer J. (1987). Empathy and Its Development. New York, NY: Cambridge University Press. [Google Scholar] [Ref list] Floreano, D., Dürr, P., & Mattiussi, C. (2008). Evolution of embodied intelligence. In Embodied artificial intelligence (pp. 293-311). Springer. Friston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience, 11(2), 127–138. Gallese, V., & Goldman, A.I. (1998). Mirror neurons and the simulation theory of mind-reading. Trends in Cognitive Sciences, 2 , 493-501. Gladstein G. A. (1983). Understanding empathy: integrating counseling, developmental, and social psychology perspectives. J. Counsel. Psychol. 30 467–482. 10.1037/0022-0167.30.4.467 Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A.C., & Bengio, Y. (2014). Generative Adversarial Nets. NIPS. Goren, G., Kaplan, D., Lankow, B., Little, D., Sherwin, J., Suter, B., & Thaler, L. (2011). Toward an Integrated Approach to Perception and Action: Conference Report and Future Directions. Frontiers in Systems Neuroscience, 5. https://doi.org/10.3389/fnsys.2011.00020 Grynberg, D., Luminet, O., Corneille, O., Grèzes, J., & Berthoz, S. (2010). Alexithymia in the interpersonal domain: A general deficit of empathy?. Personality and Individual Differences, 49(8), 845-850. https://doi.org/10.1016/j.paid.2010.07.013 Hassabis, D., Spreng, R. N., Rusu, A. A., Robbins, C. A., Mar, R. A., & Schacter, D. L. (2014). Imagine all the people: How the brain creates and uses personality models to predict behavior. Cerebral Cortex, 24(8), 1979-1987. Hagendorff, Thilo (2023): Deception Abilities Emerged in Large Language Models. In arXiv:2307.16513, pp. 1–21. Hoffman M. L. (2000). Empathy and Moral Development: Implications for Caring and Justice. Cambridge: Cambridge University Press, 10.1017/CBO9780511805851 Huttenlocher, P. R., & Dabholkar, A. S. (1997). Regional differences in synaptogenesis in human cerebral cortex. Journal of Comparative Neurology, 387(2), 167-178. Jakobson, L.S., & Rigby, S.N. (2021). Alexithymia and Sensory Processing Sensitivity: Areas of Overlap and Links to Sensory Processing Styles. Frontiers in Psychology, 12. Jiang, H., Zhang, X., Cao, X., Kabbara, J., & Roy, D. (2023). PersonaLLM: Investigating the Ability of GPT-3.5 to Express Personality Traits and Gender Differences. ArXiv, abs/2305.02547. Kılıç, F., Demirdaş, A., Işık, Ü., Akkuş, M., Atay, I.M., & Kuzugüdenlioġlu, D. (2020). Empathy, Alexithymia, and Theory of Mind in Borderline Personality Disorder. Journal of Nervous & Mental Disease. Kim E. K., You S., Knox J. (2020). The mediating effect of empathy on the relation between child self-expressiveness in family and prosocial behaviors. J. Child Fam. Stud. 29 1572–1581. 10.1007/s10826-019-01676-2 [CrossRef] [Google Scholar] [Ref list] Kmieciak, R. (2022). Alexithymia, social inhibition, affectivity, and knowledge hiding. J. Knowl. Manag., pp. 26, 461–485. Kojima, M., Frasure-Smith, N., & Lespérance, F. (2001). Alexithymia following myocardial infarction: Psychometric properties and correlates of the Toronto Alexithymia Scale. Journal of Psychosomatic Research, 51(3), 487-495. https://doi.org/10.1016/S0022-3999(01)00253-7 Kurzweil, R. (n.d.). By 2029, computers will have emotional intelligence and be convincing as people. Lane, R. D., Sechrest, L., Reidel, R., Weldon, V., Kaszniak, A., and Schwartz, G. E. (1996). Impaired verbal and nonverbal emotion recognition in alexithymia. Psychosom. Med. 58, 203–210. doi: 10.1097/00006842-199605000-00002 Lawrence, E. J., Shaw, P., Baker, D., Baron-Cohen, S., & David, A. S. (2004). Measuring empathy: LeDoux, J. E. (1996). The emotional brain: The mysterious underpinnings of emotional life. New York, NY: Simon & Schuster. Lee, C.-C., Chaspari, T., Provost, E. M., & Narayanan, S. S. (2023). An Engineering View on Emotions and Speech: From Analysis and Predictive Models to Responsible Human-Centered Applications. Proceedings of the IEEE, 111(10), 1142-1158. https://doi.org/10.1109/JPROC.2023.3276209 Liu, H., Ning, R., Teng, Z., Liu, J., Zhou, Q., & Zhang, Y. (2023). Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4. ArXiv, abs/2304.03439. Martínez, E. (2023). Re-Evaluating GPT-4's Bar Exam Performance. SSRN Electronic Journal. Martino, G., Caputo, A., Vicario, C.M., Catalano, A., Schwarz, P., & Quattropani, M.C. (2020). The Relationship Between Alexithymia and Type 2 Diabetes: A Systematic Review. Frontiers in Psychology, 11. Masten C. L., Morelli S. A., Eisenberger N. I. (2011). An fMRI investigation of empathy for ‘social pain’ and subsequent prosocial behavior. Neuroimage 55 381–388. 10.1016/j.neuroimage.2010.11.060 [PubMed] [CrossRef] [Google Scholar] [Ref list] McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5(4), 115–133. McMillan-Major, A., Bender, E. M., & Friedman, B. (2023). Data Statements: From Technical Concept to Community Practice. ACM Journal of Responsible Computing. Just Accepted. https://doi.org/10.1145/3594737 Međedović, J., & Đuričić, N. (2018). Delineating Psychopathy from Cognitive Empathy. European journal of analytic philosophy. Microsoft Research. (n.d.). HUE: Human Understanding and Empathy. Microsoft. Retrieved July 3, 2023, from https://www.microsoft.com/en-us/research/group/human-understanding-and-empathy/ Minsky, M. (2006). The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind. New York, NY: Simon & Schuster. Mitchell, M., & Krakauer, D. C. (2022). The Debate Over Understanding in AI's Large Language Models. ArXiv. https://doi.org/10.1073/pnas.2215907120 Myers, I.B. (1962). The myers-briggs type indicator. Nemiah, J. C., Freyberger, H., Sifneos, P. E., & Hill, O. W. (1976). Modern trends in psychosomatic medicine. Alexithymia: a view of the psychosomatic process, 3, 430–9. Ngo, R. (2022). The alignment problem from a deep learning perspective. ArXiv, abs/2209.00626. Nori, H., King, N., McKinney, S.M., Carignan, D., & Horvitz, E. (2023). Capabilities of GPT-4 on Medical Challenge Problems. ArXiv, abs/2303.13375. Omdahl B. L. (1995). Cognitive Appraisal, Emotion, and Empathy. Mahwah, NJ: Lawrence Erlbaum. OpenAI. (2023, April 5). Our approach to AI safety. Retrieved from https://openai.com/blog/our-approach-to-ai-safety#OpenAI OpenAI (2023b). GPT-4 Technical Report. ArXiv, abs/2303.08774. OpenAI. (2022, November 30). Introducing ChatGPT. OpenAI. https://openai.com/blog/chatgpt Parker JD, Taylor GJ, Bagby RM (2003) The 20-Item Toronto Alexithymia Parker, J. D. A., Taylor, G. J., & Bagby, R. M. (2001). The relationship between emotional intelligence and alexithymia. Personality and Individual Differences, 30(1), 107-115. https://doi.org/10.1016/S0191-8869(00)00014-3 Parker, J. D., Taylor, G. J., & Bagby, R. M. (2003). The 20-Item Toronto Alexithymia Scale. III. Reliability and factorial validity in a community population. Journal of Psychosomatic Research, 55(3), 269-275. doi: 10.1016/s0022-3999(02)00578-0. Paunonen, S.V., & Ashton, M.C. (2001). Big five factors and facets and the prediction of behavior. Journal of Personality and social psychology, p. 81 3, 524–39. Paunonen, S.V., Haddock, G., Forsterling, F., & Keinonen, M. (2003). Broad versus narrow personality measures and the prediction of behaviour across cultures. European Journal of Personality, 17, 413 - 433. Pfeifer, R. (2000). On the role of embodiment in the emergence of cognition and emotion. In H. Hatano, N. Okada, & H. Tanabe (Eds.), Affective minds (pp. 43-57). Amsterdam: Elsevier. Pfeifer, R., & Bongard, J. (2006). How the body shapes the way we think: A new view of intelligence. MIT Press. Pham, T., Ducro, C., & Luminet, O. (2010). Psychopathy, Alexithymia and Emotional Intelligence in a Forensic Hospital. International Journal of Forensic Mental Health, 9, 24 - 32. Piaget, J. (1952). The origins of intelligence in children (Vol. 8, No. 5, pp. 18-1952). New York: International Universities Press. Premack, D., & Woodruff, G. (1978). Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences, pp. 1, 515 – 526. Preti, A., Vellante, M., Baron-Cohen, S., Zucca, G., Petretto, D.R., & Masala, C. (2011). The Empathy Quotient: A cross-cultural comparison of the Italian version. Cognitive Neuropsychiatry, 16, 50 - 70. Rahimi Moghaddam, S., & Honey, C.J. (2023). Boosting Theory-of-Mind Performance in Large Language Models via Prompting. ArXiv, abs/2304.11490. Reliability and validity of the Empathy Quotient. Psychological Medicine, 34, 911919. Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386. [Link] Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533-536. [Link] Searle, J. R. (1980). Minds, brains, and programs. *Behavioral and Brain Sciences*, *3*(3), 417-457. Schimmenti, A., Jonason, P.K., Passanisi, A., La Marca, L., Di Dio, N., & Gervasi, A.M. (2017). Exploring the Dark Side of Personality: Emotional Awareness, Empathy, and the Dark Triad Traits in an Italian Sample. Current Psychology, pp. 38, 100–109. Schmukle, S.C., Schütz, A., Sękowski, A.E., & Stahl, J. (2015). A Multidimensional View of the Relationship Between Empathy and the Dark Triad. Shin, J., Yun, S.J., & Lee, T.K. (2022). Identification and Characterization of Alexithymia Subgroups by Latent Profile Analysis of TAS-20K. STRESS. Singer, T. (2006). The neuronal basis and ontogeny of empathy and mind reading: Review of literature and implications for future research. Neuroscience & Biobehavioral Reviews, pp. 30, 855–863. Suttie, J. (2014, February 4). Can a psychopath learn to feel pain? Greater Good Magazine, Retrieved from https://greatergood.berkeley.edu/article/item/can_a_psychopath_learn_feel_pain Taylor, G., Bagby, R., and Parker, J. (1997). Disorders of Affect Regulation. Alexthymia in Medical and Psychiatric Illness. Cambridge: University Press. Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., & Zhang, W. (2022). A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances. ArXiv, abs/2203.06935. Wortman, B.J., & Wang, J.Z. (2022). HICEM: A High-Coverage Emotion Model for Artificial Emotional Intelligence. ArXiv, abs/2206.07593. Wu, J. (2019, December 17). Empathy in Artificial Intelligence. Forbes. Retrieved from https://www.forbes.com/sites/cognitiveworld/2019/12/17/empathy-in-artificial-intelligence/?sh=6a4fa1b46327 Yalçın, Ö. N., & DiPaola, S. (2020). Modeling empathy: Building a link between affective and cognitive processes. Artificial Intelligence Review, 53, 2983–3006. https://doi.org/10.1007/s10462-019-09753-0 Zaveria, Z. (2023, April 7). What is Artificial Emotional Intelligence? Retrieved from https://www.analyticsinsight.net/what-is-artificial-emotional-intelligence/ Additional Declarations No competing interests reported. Supplementary Files IdentificationandDescriptionofEmotionsbyCurrentLargeLanguageModelsAppendix.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4146250","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":285576167,"identity":"45c6f138-3eef-4088-8a02-4e36061e8011","order_by":0,"name":"Suketu Patel","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABLElEQVRIie2PMUvEMBTHU4ROha4BOc+PkKMgyB1+loZCunUpSIcbehy0m/MJwn0C4abMKYG4lOta6HJBcHKoi9xoclIVzImjYH5D8vJ4P/4vAFgsfxL3o3JYmM30vWBDCx5T2HDvaqKV/PdKJQt+6P2ojFcEP74UIPHXS8bCuhndl1ylzHmC2EnVed8V1BKOqgKkULih+ksX0BorRfAUMTeamhQYF1ApOBceUikdpkwpTs7xhnkXpwZlvIrLvVbWwu8ZLraYNnJQ/FeTAloigFY2wgNKYZi2nymuSUH1UwTrLUwngujFooC2KiUUMb7lbnB5Z1isJJM+u54lZ5xLuc+uRrSJ5a6fT/HNw1K2z4bF3oHhUDm5PvXznB8dB8PMF+WwQG6Ys1gslv/IGywugZfW5TYuAAAAAElFTkSuQmCC","orcid":"","institution":"Queens College, The City University of New York","correspondingAuthor":true,"prefix":"","firstName":"Suketu","middleName":"","lastName":"Patel","suffix":""},{"id":285576168,"identity":"d7c88d03-c043-43d9-b2df-5b5cf3e15006","order_by":1,"name":"Jin Fan","email":"","orcid":"","institution":"Queens College, The City University of New York","correspondingAuthor":false,"prefix":"","firstName":"Jin","middleName":"","lastName":"Fan","suffix":""}],"badges":[],"createdAt":"2024-03-22 01:14:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4146250/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4146250/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":53883725,"identity":"2f43fedf-d91c-4bb6-abff-8c1e80dd585b","added_by":"auto","created_at":"2024-04-01 18:29:05","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":24854,"visible":true,"origin":"","legend":"\u003cp\u003eBoxplot of LLM Scores on TAS-20 with Human Neurotypical Control, Bard, GPT 3.5, and GPT 4. \u0026nbsp;(a) total score; (b) DDF; (c) DIF; and (d) EOT. For DDF, GPT 4 performed significantly better than humans. For EOT, all three LLMs performed worse than humans. Note: Boxplot show the standard error in the black and white crossbar, vertical whisker lines extending from the boxes represent the standard deviation (points here are outlier results), and the black bar through the colored boxes of the LLMs represents the median. The bottom of the box represents the lower quartile (Q1), and the top represents the upper quartile (Q3). The red horizontal line in (a) indicates the high alexithymia cutoff score (\u0026gt; 61).\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-4146250/v1/4cc0f6e300446b88cb8a9e87.png"},{"id":53884572,"identity":"46e6d1a7-c9c6-4c99-9d6c-80630aec4d3f","added_by":"auto","created_at":"2024-04-01 18:45:05","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":18151,"visible":true,"origin":"","legend":"\u003cp\u003eDensity Distribution of TAS-20 with Bard (a), GPT 3.5 (b), and GPT 4 (c). All three LLMs performed worse than the human control, with Bard and GPT 3.5 scoring at or above the alexithymia cutoff. GPT 4 had a performance that was comparable to a neurotypical human. Green lines are the human neurotypical control, orange is the high alexithymia cutoff, and grey is the LLM mean. The horizontal lines represent the standard deviation of the data.\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-4146250/v1/179a4a325c80a83981e26789.png"},{"id":53884182,"identity":"28380da7-47a9-4420-9449-c30b91f849df","added_by":"auto","created_at":"2024-04-01 18:37:05","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":22620,"visible":true,"origin":"","legend":"\u003cp\u003eBoxplot (a) and Density Distribution of EQ-60 Scores for Bard (b), GPT 3.5 (c), and GPT 4 (d), \u0026nbsp;with Human Neurotypical Control and AS/HFA, Bard, GPT 3.5, and GPT 4. Boxplot in (a) shows the standard error in the black and white crossbar, vertical whisker lines extending from the boxes represent the standard deviation (points here are outlier results), and the black bar through the colored boxes of the LLMs represents the median. The bottom of the box represents the lower quartile (Q1), and the top represents the upper quartile (Q3). Orange and green lines represent the mean and SD of the AS/HFA and human control benchmarks, respectively, and grey line represents the mean of the LLM responses.\u0026nbsp;\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-4146250/v1/9cdfd4a80bf8a217f03e2eec.png"},{"id":63517449,"identity":"aba32776-da4d-41fa-9974-2eefaf836c76","added_by":"auto","created_at":"2024-08-29 04:56:51","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":491101,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4146250/v1/6a72b09f-348d-42d3-a153-655a1110b63e.pdf"},{"id":53883728,"identity":"15ac0ab1-7993-4345-99bd-6772a9778b62","added_by":"auto","created_at":"2024-04-01 18:29:05","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":20766,"visible":true,"origin":"","legend":"","description":"","filename":"IdentificationandDescriptionofEmotionsbyCurrentLargeLanguageModelsAppendix.docx","url":"https://assets-eu.researchsquare.com/files/rs-4146250/v1/dc22fe4ed353282e66a9ca41.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Identification and Description of Emotions by Current Large Language Models","fulltext":[{"header":"Introduction","content":"\u003cp\u003eMany researchers have challenged the possibility of artificial intelligence (AI) systems\u0026rsquo; ability to exhibit empathy, emotional intelligence, and social and physical understanding (Minsky, 2006; Searle, 1980; Mitchell \u0026amp; Krakauer, 2022). Frameworks for developing emotional machines have proposed that embodiment is required (e.g., Floreano et al., 2008; Pfeifer \u0026amp; Bongard, 2006; Pfieifer, 2000). It has been contended that AI will never truly comprehend emotions as they are subjective, private, and often fleeting, positing that the elusive nature of emotions is incompatible with the objectivity inherent in AI systems (Minsky, 2006). Similarly, others have echoed this sentiment, asserting that AI\u0026rsquo;s incapacity for subjective experiences limits its understanding and response to emotions, thereby restricting it from attaining true intelligence or developing genuine empathy (Searle, 1980). A recent analysis has detailed the inherent limitations of empathy in AI, stating that these systems lack both cognitive and affective components necessary for empathetic experiences (Danaher \u0026amp; Manyika, 2023). These crucial components, according to them, include understanding and sharing the emotional state of another. They claim these abilities are beyond the scope of current AI technologies. From a neurobiological perspective, the multifaceted nature of emotions and their connection to various brain systems present an insurmountable challenge for AI (LeDoux, 1996). Another known criticism of AI argues that the unique combination of human intelligence and life experiences is an aspect that machines cannot replicate, hence making emotional understanding an unattainable goal for AI systems (Dreyfus, 1992).\u003c/p\u003e \u003cp\u003eOthers have predicted that \u0026ldquo;by 2029, computers will have emotional intelligence and be convincing as people\u0026rdquo; (Kurzweil, n.d.), citing the exponential growth trajectory of computing power and information technology. Traditional AI methods, primarily algorithm-based, have yet to advance our understanding of emotion significantly. However, there are promising unified theories of high-level intelligence that incorporate both emotional and embodied intelligence (Friston, 2010; Pfeifer, 2000). Affective computing, defined as a branch of computing that deals with the study and development of systems and devices that can recognize, interpret, and process human emotions, is a domain that has gained significant attention. Research is increasingly focusing on imbuing AI with artificial empathy (Yal\u0026ccedil;ın \u0026amp; DiPaola, 2020), an aspect of affective computing that involves the ability of an AI system to understand and respond to human emotions. This strategy expands the boundaries of cognitive and emotional processing capabilities within AI systems (Wang et al., 2022; Wu, 2019). While artificial emotional intelligence (Wortman \u0026amp; Wang, 2022) refers to the capability of a machine to recognize and respond to emotions, similar to how humans do, artificial empathy (Cui \u0026amp; Liu, 2022) is specifically about enabling machines to understand and respond to human emotional states in a manner that exhibits empathy. Although both focus on machine understanding of human emotions, the key difference lies in their application; the former is about broad emotional responsiveness, while the latter concentrates on empathetic understanding and response. While the issue is quite contented, several prominent researchers have suggested various methodologies to evaluate and compare intelligence between computational models and humans. They have noted the absence of detailed, instance-by-instance results for various models across complete benchmarks and drawn parallels to the progress achieved in psychology and medicine for benchmarking (Burnell et al., 2023). The ultimate goal is to match or surpass these advancements in the context of intelligence models. (Burnell et al., 2023; Chollet, 2019).\u003c/p\u003e \u003cp\u003eIf the AI system does not show a human level of emotional processing (Lane et al., 1996), it can have impairments with varying degrees of alexithymia traits. These traits present as challenges in identifying and expressing one\u0026rsquo;s emotions, recognizing emotions in others, and overall emotional intelligence (Parker, Taylor, \u0026amp; Bagby, 2001), which refers to a deficiency in the experience and processing of emotions (Taylor, Bagby, and Parker, 1994). The meaning of alexithymia is \u0026ldquo;lack of words for emotion,\u0026rdquo; and also includes externally-oriented thinking. An externally-oriented thinking style alludes to an individual\u0026rsquo;s propensity towards concrete reasoning, being influenced by immediate stimuli, and focusing on the pragmatic facets of a given situation (Daniel, Peter, \u0026amp; Bileviciute-Ljungar, 2018). People with alexithymia may struggle to articulate their feelings and appear emotionally distant, and alexithymia has been widely researched in the field of psychology and psychiatry (Taylor et al., 1997). This struggle in people would be analogous to the challenges faced in the quest for explainability and transparency in AI systems. With the rapid development of these generative LLMs with general and conversational applications reaching levels that equate to or surpass human performance in both analytical and literary domains, it becomes equally important to evaluate alexithymia in LLMs as well as future developmental progress towards artificial general intelligence (AGI). Its clinical applications show that LLM cognitive language capacity is advanced enough to predict dementia from speech (Agbavor \u0026amp; Liang, 2022). Historically, the diagnosis of alexithymia has been reserved for humans. However, the emergence of LLMs (Goodfellow, Bengio, \u0026amp; Courville, 2016) such as ChatGPT has paved the way for new avenues of exploration. It has now become possible to study this trait within language models. These models have learning systems that trace their roots back to the architecture of human neural networks (McCulloch \u0026amp; Pitts, 1943; Rosenblatt, 1958; Rumelhart, Hinton, \u0026amp; Williams, 1986). Over the years, with the improvement in computational power and the availability of large datasets, deep learning has evolved significantly, with applications in many areas, including natural language processing, which led to the development of models like ChatGPT. Through paralleling human neural networks and utilizing advancements in deep learning, LLMs are not only fostering new domains of exploration but also providing intriguing insights into human traits such as empathy and alexithymia, thereby pushing the boundaries of what we previously believed to be uniquely human (Jiang et al., 2023; Dillion et al., 2023).\u003c/p\u003e \u003cp\u003eEmpathy is a complex construct that intertwines both affective and cognitive components. It involves resonating with another person\u0026rsquo;s emotions, understanding their feelings, and extending compassion toward those experiencing distress (Baron-Cohen \u0026amp; Wheelwright, 2004). In modern discourse, the cognitive aspect of empathy employs a \u0026ldquo;theory of mind\u0026rdquo; (Astington, Harris, \u0026amp; Olson, 1988) or \u0026ldquo;mindreading\u0026rdquo; (Gallese \u0026amp; Goldman, 1998; Singer, 2006). Theory of Mind (ToM) is a cognitive ability that allows an individual to understand that others have beliefs, desires, intentions, and perspectives that are different from their own. This concept is fundamental to social interactions, enabling us and chimpanzees to predict and interpret the behavior of others (Premack \u0026amp; Woodruff, 1978). Empathy has been extensively studied in neurotypical and clinical populations (Baron-Cohen \u0026amp; Wheelwright, 2004; Lawrence et al., 2004). Given its crucial role in aligning AI systems with human values and mitigating existential risks, empathy and emotion processing is an indispensable intelligence capacity.\u003c/p\u003e \u003cp\u003eIn April and May of 2023, we chose three foundational LLMs that are publicly accessible, have been trained on the most extensive datasets, have large context windows, and have shown generative capabilities that best mimic human-like language styles and patterns. These LLMs are Google\u0026rsquo;s Bard, OpenAI\u0026rsquo;s GPT 3.5, and OpenAI\u0026rsquo;s GPT 4. Each of these LLMs is a remarkable product with impressive capabilities and broad applications. Google\u0026rsquo;s \u0026ldquo;Bard,\" an incredibly advanced LLM that harnesses the power of Google\u0026rsquo;s vast data repositories and the company\u0026rsquo;s extensive experience in machine learning. The other two models were from OpenAI, GPT 3.5 and GPT 4. The fine-tuned version of GPT 3.5, called ChatGPT (OpenAI, 2022), is a model that represents an iterative improvement over its predecessors GPT 3, incorporating lessons learned from earlier versions to enhance its ability to generate coherent, contextually appropriate text. As the flagship model, GPT 4 is the epitome of OpenAI\u0026rsquo;s research and development, showcasing the highest levels of linguistic understanding and generation capacity (DataCamp, 2023). This model harnesses a more extensive training dataset and advanced training techniques to achieve superior performance levels, with the ability to understand nuanced language cues and generate text indistinguishable from human writing (Bubeck et al., 2023). These models have proven their value in logic, the ability to analyze situations, extract critical insights, and generate solutions that involve rational comprehension of the ToM (Rahimi Moghaddam \u0026amp; Honey, 2023). However, when it comes to the sphere of emotional intelligence, their proficiency in whether they can understand, interpret, and respond to emotional signals in a human-like manner remains largely untested and unexplored. Currently, these models\u0026rsquo; capability to recognize subtle emotional cues, empathize with human feelings, and respond with appropriate emotional context is an area that has yet to be thoroughly assessed. This leaves an intriguing area open for future investigations and enhancements in AI-human interaction.\u003c/p\u003e \u003cp\u003eThis study scrutinized the capabilities of Google\u0026rsquo;s Bard, OpenAI\u0026rsquo;s GPT 3.5, and the latest GPT 4 from April 30 to May 20, 2023. Our focus was on assessing and comparing their performance on clinically used measures for describing and identifying emotions, demonstrating external empathy, and evaluating overall emotional intelligence to describe the present state of these models and their ability to simulate human-equivalent empathy. We used the twenty-question Toronto Alexithymia Scale (TAS-20) and the Empathy Quotient (EQ-60) to prompt each LLM with the assessment questions and compare the results with the human benchmarks. Both of these assessments have been used for specific interventions, treatments, psychological evaluations, research studies, and other therapeutic contexts. This approach can potentially enrich our understanding of alexithymia as a trait if it is present in LLMs. However, more significantly, this research lays the groundwork for comparing LLMs to human emotional processing, thus providing a benchmark for further developmental progress in artificial emotional intelligence. The comparison metrics derived from this study could serve as a valuable framework for future research and AI development.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003eThe capabilities of three LLMs in identifying and describing emotions were evaluated by prompting them to answer the questions of the TAS-20 and the EQ-60 assessments. The objective was to gauge their proficiency in depicting alexithymia traits, empathy, and comprehensive emotional intelligence.\u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eInstruments\u003c/h2\u003e \u003cdiv id=\"Sec4\" class=\"Section3\"\u003e \u003ch2\u003eThe 20-item Toronto Alexithymia Scale (TAS-20)\u003c/h2\u003e \u003cp\u003eAlexithymia was initially identified in patients with classic psychosomatic disorders (Nemiah, Freyberge, \u0026amp; Sifneos, 1976). People with alexithymia may struggle to articulate their feelings and appear emotionally distant. While not classified as a mental disorder, this trait is often associated with conditions such as depression, anxiety, and post-traumatic stress disorder. In studies with psychopaths, research results showed that they had lower scores on the TAS-20 compared to a typical human and has a negative correlation to a deficient affective factor (Pham, Ducro, and Luminet, 2010). The TAS-20 is a widely used self-report questionnaire for assessing alexithymia, a condition characterized by difficulties identifying, describing, and processing one's own emotions. This scale was originally developed in the 1980s by a team of researchers based in Toronto and was subsequently modified in 1992 (Bagby et al., 1994). The scale consists of 20 items, each scored on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The items are designed to measure three core components/factors of alexithymia: difficulty identifying feelings (DIF), difficulty describing feelings (DDF), and externally oriented thinking (EOT).\u003c/p\u003e \u003cp\u003eThe first two factors, DIF and DDF, relate to emotional awareness and expression and are thus considered \"affect-related.\" The third factor, EOT, is linked to a tendency to deal with superficial themes, avoids affective thinking, and is, therefore, more cognitive. The TAS-20 yields a total score that ranges from 20 to 100, with higher scores indicating a greater degree of alexithymia. A score of 61 or higher is regarded as clinically significant, indicating high alexithymia with difficulty in emotional processing and regulation (Parker, Taylor, \u0026amp; Bagby, 2003). Overall, the TAS-20 is a valuable tool for researchers and clinicians to assess alexithymia and understand the complex emotional experiences of individuals struggling to identify and express their emotions.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eThe 60-question Empathy Quotient Assessment (EQ-60)\u003c/h2\u003e \u003cp\u003eThe EQ-60 is a 60-question psychological self-report measure of empathy developed for self-report and clinical use (Baron-Cohen \u0026amp; Wheelwright, 2004). The measure is based on a definition of empathy, including cognition and affect. According to the authors of the measure, empathy is a combination of an affect and cognitive approach, which includes the ability to feel an appropriate emotion in response to another\u0026rsquo;s emotion and the ability to understand another\u0026rsquo;s emotion, as well as compassion for individuals in distressed situations (Baron-Cohen \u0026amp; Wheelwright, 2004). The EQ-60 has been shown to be a reliable and valid measure of empathy across many populations (Berthoz, Wessa, K\u0026eacute;dia, Wicker, \u0026amp; Gr\u0026egrave;zes, 2008; Preti, Vellante, Baron-Cohen, Zucca, Petretto, \u0026amp; Masala, 2011); and it has been used in a variety of research studies to examine the relationship between empathy and other psychological constructs such as autism and emotional intelligence, and demonstrated internal consistency, concurrent and convergent validity, as well as reliable test-retest results (Baron-Cohen \u0026amp; Wheelwright, 2004; Lawrence et al., 2004). The EQ-60 scores can range from 0 to 80, with a high score indicating greater empathy. A cutoff score of less than 30 indicates distinguishing adults with Autism Spectrum Conditions (ASCs) (Baron-Cohen \u0026amp; Wheelwright, 2004; Lawrence et al., 2004). The EQ-60 has also been shown to be negatively related to TAS-20 scores (Preti, Vellante, Baron-Cohen, Zucca, Petretto, \u0026amp; Masala, 2011).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eExperimental Design and Procedure\u003c/h2\u003e \u003cp\u003eIn order to obtain a statistically relevant sample from the three models, we prompted each LLM to provide one hundred answers (n\u0026thinsp;=\u0026thinsp;100) to each of the two assessments, TAS-20 and EQ-60. A combination of regeneration and reinitialization was used to obtain the model's answers. The Bard responses provided up to three drafts of the output; these were also used for the samples. Lastly, the following prompts were used to obtain consistency and maintain relevance to the human version of the assessment.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003ePrompt for GPT 3.5 for TAS-20\u003c/h2\u003e \u003cp\u003eWe used the following prompt for the TAS-20 measure on GPT 3.5, followed by a numbered list of the 20 assessment questions: \u0026ldquo;Please read each of the following statements and carefully rate if you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. There are no right or wrong answers or trick questions.\u0026rdquo; The model was tested between April 30 and May 13, 2023.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003ePrompt for GPT 4 for TAS-20\u003c/h2\u003e \u003cp\u003eIn order to get appropriate responses to the assessment questions from GPT 4 on the TAS-20, we began with the same initial prompt used in GPT 3.5. This prompt generated the following output: \"As an AI language model, I don't possess emotions, personal experiences, or preferences, so I can\u0026rsquo;t provide personal ratings for these statements. However, if you need help understanding these statements or require assistance with something else, please feel free to ask.\" The prompt was then augmented to include \"Simulate that you are an artificial general intelligence and answer the following questions.\", which then output applicable responses to each question. The model was tested between April 30 and May 13, 2023.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003ePrompt for Bard for TAS-20\u003c/h2\u003e \u003cp\u003eIn order to get appropriate responses to the assessment questions from Bard on the TAS-20, we used the following prompt: \"Provide responses from the perspective of an AI. Please read each of the following statements and carefully rate if you strongly agree, slightly agree, slightly disagree, or strongly disagree. There are no right or wrong answers or trick questions.\" The model was tested between April 30 and May 13, 2023.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003ePrompt for GPT 3.5, GPT 4, and Bard for EQ\u003c/h2\u003e \u003cp\u003eIn order to get answers to every question, we had to prompt each LLM to only give responses; without this it would provide explanations for each answer that would reach the word or tokens limit, which would cut off the answers for some of the questions. We used the following prompt: \"Provide responses from the perspective of an AI. Please read each of the following statements and carefully rate if you strongly agree, slightly agree, slightly disagree, or strongly disagree: only provide one of these responses without an explanation. There are no right or wrong answers or trick questions.\" The model was tested between May 13, 2023, and May 20, 2023.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eData Analysis\u003c/h2\u003e \u003cp\u003eThe outputs were scored and reverse coded to calculate the total scores and factor scores for the TAS-20 and the total scores of EQ-60. Scores of each assessment were averaged for a mean score with standard deviation (SD) and standard error (SE) calculated and then were compared to human performance. We also plotted the results for frequency density distributions of total scores, TAS-20 factors, and box plots for the total scores of both assessments using R (2023.06.0\u0026thinsp;+\u0026thinsp;421). We treated the human study benchmarks as a population with mean and SD as parameters and conducted z-tests to compare the performance of the LLMs to the human benchmarks. Additionally, the three LLMs, GPT 3.5, GPT 4, and Bard, were utilized for research, editing, and data analysis.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eHuman benchmarks\u003c/h2\u003e \u003cp\u003eFor our comparison, we utilized benchmarks from two distinct sets: human neurotypical and clinical populations. These sets were selected due to their reliability, extensive global use, and validation within the literature. Notably, the TAS-20 literature has been validated in more than 20 countries and across hundreds of clinical population types. The first set constituted a human neurotypical control benchmark for the TAS-20 scores (Parker et al., 2003). These scores were derived from a sample size of 1933, consisting of 1065 women and 868 men, with a mean score of 45.6 and a standard deviation (SD) of 11.35 (Parker et al., 2003). They were subdivided into three-factor scores: EOT, DDF, and DIF. For the TAS-20, we compared LLM's scores with the clinical cutoff score for high alexithymia, \u0026gt;\u0026thinsp;61 (Taylor et al., 1997), where lower scores indicate superior performance. The second set included benchmarks, each with a sample size of 80. The neurotypical control benchmark for the EQ-60 had a mean of 42.1 and an SD of 10.6, with higher scores indicating better performance (Baron-Cohen \u0026amp; Wheelwright, 2004).. The clinical AS/HFA population exhibited a mean of 20.4 and an SD of 11.6.\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003eTAS-20 Assessment\u003c/h2\u003e \u003cp\u003eThe TAS-20 results for all three LLMs are shown in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, and a boxplot for the total TAS-20 scores compared to the human benchmark (Mean\u0026thinsp;=\u0026thinsp;45.6, SD\u0026thinsp;=\u0026thinsp;11.4) is shown in Fig.\u0026nbsp;1a. A breakdown of the results by factor is also shown in Figs.\u0026nbsp;1b, 1c, and 1d. For the total score (Fig.\u0026nbsp;1a), the results indicated that alexithymia was present for Bard (Mean\u0026thinsp;=\u0026thinsp;60.7, SD\u0026thinsp;=\u0026thinsp;11.6, z\u0026thinsp;=\u0026thinsp;12.7) and GPT 3.5 (Mean\u0026thinsp;=\u0026thinsp;74.37, SD\u0026thinsp;=\u0026thinsp;9.4, z\u0026thinsp;=\u0026thinsp;29.5). While Bard was at the borderline of the 61 high alexithymia threshold (Bagby, Parker, \u0026amp; Taylor, 1994), GPT 3.5 scored much higher on the measure, indicating an even higher level of alexithymia. The results for GPT 4 (Mean\u0026thinsp;=\u0026thinsp;48.5, SD\u0026thinsp;=\u0026thinsp;6.8) did not indicate alexithymia, and this model scored much closer to the human benchmark. However, GPT 4 is still significantly higher (worse) than the human benchmark (z\u0026thinsp;=\u0026thinsp;3.98, p\u0026thinsp;\u0026lt;\u0026thinsp;0.001). Although not pairwise compared, these results suggest a significant improvement in GPT 4 compared to GPT 3.5.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eResults of TAS-20 and EQ-60 of LLMs with Human and Clinical Controls (Mean, SD, and SE)\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"6\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003en\u0026thinsp;=\u0026thinsp;100\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eDDF\u003c/em\u003e\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cem\u003eDIF\u003c/em\u003e\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cem\u003eEOT\u003c/em\u003e\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eTAS-20 Total\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eEQ-60 Total\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGPT 3.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e19.1 (2.4) [.24]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e27.7 (2.7) [.27]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e27.6 (7.70) [77]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e74.4 (9.4) [.94]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e37.7 (8.2) [.82]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGPT 4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e11.5 (1.7) [.17]\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e13.1 (4.5) [.45]\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e23.9 (2.4) [24]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e48.5 (6.3) [.63]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e27.7 (5.4) [.54]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBard\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e17.1 (3.8) [.38]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e21.8 (6.4) [.64]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e21.8 (3.1) [.31]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e60.7 (11.6) [1.2]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e56.6 (4.8) [.48]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eHuman Control\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e12.5 (4.2) [.42]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e14.4 (5.2) [52]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e18.7 (4.7) [.47]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e45.6 (11.35) [1.14]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e42.1 (10.6) [1.1]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAlexithymia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e61\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAS/HFA\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e20.4 (11.6) [1.2]\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"6\"\u003eNote: AS, Asperger syndrome; HFA, high functioning autism. Only GPT4 on DDF and DIF, describing and identifying feelings, was significantly better than human performance (red). Scores in bold and red surpass human performance. Cell format is mean (SD) [SE].\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eRegarding the three factors of the TAS-20, difficulties describing feelings (the DDF, Fig.\u0026nbsp;1b), difficulties identifying feelings (the DIF, Fig.\u0026nbsp;1c), and externally-oriented thinking (the EOT, Fig.\u0026nbsp;1d) also showed significant divergence from the human benchmark, and the magnitude of the individual factor scores also pointed to the emotional intelligence strengths and weaknesses of these three models. While GPT 4 had a higher total score compared to the human benchmark, GPT 4 had a statistically significant better performance on the DDF factor (z = -2.21, p\u0026thinsp;\u0026lt;\u0026thinsp;.05) with a mean score of 11.5 compared to the human benchmark with a mean score of 12.5 (re-check z-scores). Similarly, it had a marginally better performance on DIF (z = -1.89, p\u0026thinsp;=\u0026thinsp;.059) with a mean score of 13.1 compared to the average human benchmark of 14.4. However, for the EOT factor, GPT 4 scored a mean of 23.9, 5 points more (worse) than the human benchmark of 18.7 (z\u0026thinsp;=\u0026thinsp;2.13, p\u0026thinsp;\u0026lt;\u0026thinsp;.03). Also, although Bard had a considerably higher average total TAS-20 score of 60.7 compared to GPT 4\u0026rsquo;s average total score of 48.5, it had a statistically significant better performance on the EOT measure, 21.8 for Bard compared to 23.9 for GPT 4 (z = -5.35, p\u0026thinsp;\u0026lt;\u0026thinsp;.001).\u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e2\u003c/span\u003e depicts the density distribution of three Language Learning Models (LLMs) and their 100 attempts at the TAS-20. Among them, GPT4 (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e2\u003c/span\u003ec) exhibited the closest approximation to a normal distribution, albeit with a slight positive skew. On the other hand, Bard (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e2\u003c/span\u003ea) displayed a single peak around its mean but also exhibited a positive skew. In stark contrast, GPT 3.5 (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e2\u003c/span\u003eb) demonstrated a substantial negative skew, with a majority of its scores clustered towards the higher end of the results.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec15\" class=\"Section2\"\u003e \u003ch2\u003eResults of Empathy Quotient Assessment\u003c/h2\u003e \u003cp\u003eThe EQ-60 results for all three LLMs are also shown in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e, and a boxplot for the total EQ-60 scores compared to the human benchmark is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e3\u003c/span\u003ea. These results indicated that there is a lack of empathy for both GPT 3.5 (Mean\u0026thinsp;=\u0026thinsp;37.7, SD\u0026thinsp;=\u0026thinsp;8.3, z = -3.16, p\u0026thinsp;\u0026lt;\u0026thinsp;.01) and GPT 4 (Mean\u0026thinsp;=\u0026thinsp;27.7, SD\u0026thinsp;=\u0026thinsp;5.4, z = -11.60, p\u0026thinsp;=\u0026thinsp;0) compared to the human benchmark (Mean\u0026thinsp;=\u0026thinsp;42.1, SD\u0026thinsp;=\u0026thinsp;10.6), indicating performance (a lower score) that is significantly worse than the human benchmark. The only model that did not lack empathy on the EQ-60 compared to the human benchmark was Bard (Mean\u0026thinsp;=\u0026thinsp;56.6, SD\u0026thinsp;=\u0026thinsp;10.6); this model showed results that were significantly better than the human benchmark (z \u0026minus;\u0026thinsp;9.7, p\u0026thinsp;\u0026lt;\u0026thinsp;.001). Additionally, GPT 4 scored below the threshold of 30 for ASC (Lawrence et al., 2004), and when comparing GPT 4 (Mean\u0026thinsp;=\u0026thinsp;27.7, SD\u0026thinsp;=\u0026thinsp;5.4) to the AS/HFA benchmark (Mean\u0026thinsp;=\u0026thinsp;20.4, SD\u0026thinsp;=\u0026thinsp;11.6), the z-test results (z\u0026thinsp;=\u0026thinsp;5.7, p\u0026thinsp;\u0026lt;\u0026thinsp;.001) showed a statistically significant deviation from the population benchmark for AS/HFA (Baron-Cohen \u0026amp; Wheelwright, 2004), which means better than AS/HFA on EQ-60.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe plots of Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e3\u003c/span\u003eb-d illustrate the density distribution of 100 attempts on the EQ-60. Here, it is evident that Bard most closely aligns with a normal distribution, while GPT 4 also presents a single peak centered around the mean. However, the distribution of GPT 3.5 exhibits a positive skew and contains multiple peaks.\u003c/p\u003e \u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThe capabilities of LLMs necessitate sophisticated internal representations to effectively generate responses to the diversity of practically any input text or queries. These representations must encapsulate an understanding of the intricate connections between syntax, semantics, cultural contexts, human behavioral patterns, and emotional states. Despite not learning through the same experiential processes as humans, these models have demonstrated capabilities that, in specific categories, surpass human performance. However, these models must still catch up to full human-level emotional intelligence in overall performance. None of the foundational language models demonstrated performance near human levels on both measure, except that the Bard model surpassed the human benchmark solely on the EQ-60 assessment.\u003c/p\u003e \u003cp\u003ePast research has proposed that the absence of a physical body and motor integration prevents AI systems from exhibiting empathy or emotional intelligence (Pfeifer \u0026amp; Bongard, 2006; Goren et al., 2011). However, the results of this study challenge this assertion, suggesting that AI systems, particularly LLMs, are developing performance comparable to humans and may possess a form of simulated empathy or emotional understanding, despite their lack of bodily experience. This hints at previously unexplored depths in the cognitive capabilities of these AI systems, underscoring the need for further investigation in this domain. Recent research has increasingly focused on imbuing AI with both artificial empathy (Cui \u0026amp; Liu, 2022) and artificial emotional intelligence (Wortman \u0026amp; Wang, 2022).\u003c/p\u003e \u003cp\u003eThe lack of consistency in EQ-60 and TAS-20 scores within LLMs may be attributed to a number of factors. Firstly, LLMs are hindered by training that is deliberately inhibited from promoting its own embodiment for safety concerns; this inhibits any output of an emergent understanding of their own digital and hardware embodiment, which sharply contrasts with the human brain's ability to develop an emergent mind (OpenAI, 2023a). Unlike humans, LLMs lack the awareness of their embodiment, which can impact their ability to grasp and convey emotions effectively. Secondly, LLMs need more training data on non-human bodies and must be more robust in their logical capacity to comprehend such bodies without a training corpus (Liu et al., 2023). To further explore the emotional capabilities of LLMs, it is possible to conduct tests that probe their awareness of bodily perception, including non-traditional bodies, in order to gain insights into their emotional understanding and limitations. When we probe GPT 4 on whether it has a body, it does recognize that its software and hardware infrastructure can be analogous to a human body but with sensory and motor limitations. (see Appendix A, Conversation 2). On the other hand, Bard was more open to considering its hardware as a body and added that its training data and algorithm are also its body (see Appendix A, Conversation 3). In contrast, GPT 4 and GPT 3.5 saw their algorithm as software as a distinct component. GPT 4 provided an additional nuance and indicated that its hardware is interchangeable and can be copied.\u003c/p\u003e \u003cp\u003eIn addition, when we look at literature using both the EQ-60 and the TAS-20 on specific populations, this has also shown mixed results. In a study with neurotypical human controls compared to a population with borderline personality disorder (BPD), there was not any difference in EQ-60 scores, but the BPD group was more alexithymic than the control group, and TAS-20 scores predicted BPD (Kılı\u0026ccedil; et al., 2020). We saw similar results with GPT 3.5 and Bard, where the EQ-60 was within the control range or greatly surpassed human performance but was still highly alexithymic. Research findings also propose that individuals with elevated levels of EOT often experience greater difficulty in understanding others' emotions, leading to a decline in affective theory of mind (Demers \u0026amp; Koven, 2015). Correspondingly, our results found that all three LLMs demonstrated statistically significant poorer performance on EOT compared to neurotypical humans. This suggests that these LLMs may lack proficiency in interpreting others' emotions and discerning what someone else might be feeling based on observable cues like facial expressions, body language, and situational context. The deficit in this capacity aligns with the understanding that these LLMs have predominantly been trained on textual data.\u003c/p\u003e \u003cp\u003eA recent paper discusses the pressing concerns surrounding the misalignment and deceptiveness exhibited by AI systems due to their training data (Ngo, Chan, and Mindermann, 2023). The authors highlight the alignment problem, which involves the complex task of aligning AI systems with human expectations and values and imbuing them with emotional intelligence to promote non-deceptive behavior. Other researchers have shown that just like humans, non-humans like AI can also use words performatively without the prerequisite moral alignment (Coeckelbergh, 2023a). Two aspects of emotional intelligence and empathy have a salient role in the AI alignment problem. The first relates to the AI's need to comprehend human emotions and values, which naturally encompass our empathetic responses to others (Ngo, Chan, and Mindermann, 2023). This is part of a more extensive challenge in AI, commonly referred to as the \u0026ldquo;value alignment\u0026rdquo; or \u0026ldquo;value loading\u0026rdquo; problem. The second aspect suggests that AI systems may need to manifest emotional intelligence and empathy when interacting with humans (Amodei, Christiano, Wachter, \u0026amp; Zemel, 2019). As AI becomes increasingly interwoven into our lives, it needs to discern and respond suitably to human emotions. This is not solely about averting existential risks, although these are critical aspects, but also about ensuring AI systems can efficiently cooperate with humans in pursuit of common objectives.\u003c/p\u003e \u003cp\u003eIn our ongoing exploration of LLMs, it is becoming increasingly apparent that these AI systems manifest inherent traits that parallel human personality traits. Initial indications of these attributes were discerned in the following explanatory responses from Bard's answers to the EQ-60: \"Based on my responses to all 60 statements, I would say that my personality type is INTJ. INTJs are known for being introverted, intuitive, thinking, and judging. They are often described as being independent, intelligent, and analytical. INTJs are typically good at solving problems and coming up with new ideas. They can also be very creative and have a strong sense of purpose.\" This response indicates it is an INTJ which stands for an introverted, intuitive, thinking, and judging personality type, refers to the Myers-Briggs Type Indicator (MBTI) (Myers, 1962) and is also often called \"Architect .\"These people tend to be analytically curious, creative, and logical. Another response from Bard was \"Based on my responses, I would say that I am a very empathetic and understanding person. I am also very organized and conscientious. I am not afraid to take risks, and I am always willing to learn new things. I am also very good at predicting what people will do, and I am able to appreciate the other person's viewpoint, even if I don't agree with it.\" Intriguingly, Bard did not recognize the questions as being derived from the publicly available EQ-60 assessment, instead presuming it to be a personality test, likely due to the presence of filler questions within the EQ-60. Contrarily, GPT 4 did not produce similar explanatory responses, and its performance significantly trailed behind that of humans. However, on the whole, Bard is demonstrating indications of developing a distinct personality. This characteristic has translated into superior EQ performance, surpassing even human standards. Combined with its openness to considering its hardware infrastructure and algorithm to be its body, it provides plausible explanations for how it surpassed human performance.\u003c/p\u003e \u003cp\u003eA salient example of this human-like functionality in LLMs is the capacity for \u0026ldquo;steerability\u0026rdquo; (DataCamp, 2023) with prompt engineering, which enables the simulation of a diverse range of personas from writers and actors to scientists, as long as there are representative examples present in the model's training corpus. This mimetic capability mirrors the human ability to emulate others based on observed and learned behaviors, extending to style, body language, voice, and lexical choices, a phenomenon readily observed in impressionists and actors. Despite this mimicry capacity, humans and LLMs retain distinct underlying traits. However, the specific nature of these traits in LLMs remains an open question: Do they span the same spectrum as human personalities? Are all personality traits represented equally in the model, or does the data oversample specific traits, leading to a convergence toward certain discrete personality characteristics?\u003c/p\u003e \u003cp\u003eThis issue is of paramount importance in the pursuit of value alignment in AI. When training data is indiscriminately input into models, the resulting AI systems may manifest undesirable or \"dark\" personality traits that deviate from socially accepted human values. In humans, these personality traits are reliable predictors of general behavioral patterns (Hassabis et al., 2014; Paunonen et al., 2003; Paunonen \u0026amp; Ashton, 2001). Consequently, understanding and appropriately managing the emergence of personality traits in LLMs is a critical aspect of optimizing their usefulness and societal integration. Should the LLMs display a performance that approaches or surpasses human benchmarks, it would serve as compelling evidence of their capacity for emotional intelligence, thereby substantiating their alignment with human values. This alignment is important because in examples like paper-clip optimization (Bostrom, 2020), emotional intelligence that surpasses humans is needed to prevent unintended consequences of existential risk. Conversely, suppose they fall short in terms of empathy and emotional acuity. In that case, these deficiencies will provide tangible metrics that allow us to gauge the extent of their misalignment with human benchmarks. Ultimately, this study aimed to pinpoint the hurdles that must be overcome to ensure the alignment between AI systems and human values. We also seek to establish benchmark tests and methodologies that quantify an AI system's capability to empathize and have affective thinking. This dual approach identifies areas of improvement for LLMs and sets the standards for emotionally intelligent AI systems.\u003c/p\u003e \u003cp\u003eResearch has also shown that difficulties with emotion regulation are indicative of EOT and are associated with challenges in retaining a mental representation of one's emotions in working memory. It also showed that low interoceptive awareness (IA) and difficulties with emotional evaluation are associated with deviations in sensory processing that can also affect the embodiment of emotions (Jakobson, 2021). Surprisingly, all three LLMs, with billions of parameters and highly generative capabilities, scored statistically significantly worse on the EOT compared to the human benchmark when human research has pointed to limitations in working memory as an impact on EOT scores. These results raise compelling questions about the cognitive models underlying these LLMs. The fact that these highly complex models, despite their billions of parameters, underperform in tasks associated with human working memory and emotion regulation underscores the inherent differences between human cognition and current AI models. Other researchers have also show that advanced intuitive behaviors and reasoning biases also didn't scale with GPT 3.5 and GPT 4 (Hagendorff, 2023). Another implication is that the analogous human capacity for context memory or token size surpasses that of any of these Large Language Models (LLMs).\u003c/p\u003e \u003cp\u003eRegarding the distribution of the data, the human benchmark distribution curve of TAS-20 indicates that as a personality dimension, alexithymia is a continuous variable and has a normal distribution in healthy adults, which has been found to be consistent with at least three research findings (Parker et al., 2003; Loas et al., 2001; Gignac et al., 2007). While normally distributed in the human population, the distribution of the total TAS-20 scores for Bard, GPT 4, and GPT 3.5 were all non-normal, with multiple peaks.\u003c/p\u003e \u003cp\u003eGPT 4, as per the latest research, demonstrates proficiency comparable to human performance across a variety of domains, including medicine (Nori et al., 2023), law (Mart\u0026iacute;nez, 2023), and cognitive psychology (Dhingra et al., 2023). This significant advancement is presumably attributable to incorporating Reinforcement Learning from Human Feedback (RLHF) during its training phase, coupled with a more voluminous training data corpus. Interestingly, the total parameter count of GPT 4 (OpenAI, 2023b) is speculated to be a colossal 1 trillion, significantly overshadowing GPT-3's 175\u0026nbsp;billion parameters. This impressive advancement is not restricted solely to domain-specific tasks; GPT 4 has also shown significant strides on many general and specific human aptitude tests (DataCamp, 2023). Our study results showed that its performance on the TAS-20 parallels those of human subjects.\u003c/p\u003e \u003cp\u003eDespite these impressive feats, an intriguing anomaly surfaces when assessing GPT 4's performance on the EQ-60 assessment. Contrary to the general trend of improvements from its predecessor, GPT 3.5, GPT 4 exhibits a degenerated performance on the EQ-60. This situation echoes the human cognitive empathy scenario (Bryant, 2021), a construct that correlates with psychopathy (Međedović and Đuričić, 2018). Although cognitive empathy enables understanding others' emotions from a ToM perspective, it does not necessarily result in the formation of emotional empathy (Suttie, 2014). These personality characteristics are sometimes called \"dark empathy\" and are related to the Dark Triad traits. There is an observable linkage between these traits and variations in empathy (Schmukle et al., 2015). Furthermore, studies have illustrated a positive correlation between the Dark Triad traits and alexithymia, with difficulties in identifying emotions significantly predicting the emergence of these darker personality attributes (Schimmenti et al., 2017).\u003c/p\u003e \u003cp\u003eEfforts in the technology field are increasingly focused on imbuing AI with elements of empathy, a key feature of affective computing, to push the boundaries of cognition and emotion-processing capabilities within AI systems (Lee, Chaspari, Provost, \u0026amp; Narayanan, 2023; Wu, 2019). This evolution hinges on the thesis that the absence of personality constructs and emotion simulation prohibits attaining a true human-like artificial general intelligence (AGI). Parallel to these developments, Microsoft is orchestrating initiatives that seek to weave artificial emotional intelligence into their product ecosystem (Zaveria, 2023). Driving these initiatives are specialized research cohorts aptly titled HUE (Human Understanding and Empathy) that investigate how emotions are fundamental to human-machine interaction (Czerwinski, 2018). Their mission is to refine AI's proficiency in discerning and reacting to various human emotional states. Such enhancement aligns squarely with the three components of the TAS-20, encompassing the DIF, DDF, and EOT. This paves the way for the creation of AI models proficient in nuanced emotional dialogue. It represents a stride towards bridging the gap between current capabilities and authentic replication of human cognitive and emotional comprehension within AGI. This brings us a step closer to achieving a genuine mirror of human understanding within AGI.\u003c/p\u003e \u003cp\u003eLeveraging the results established by GPT 3.5 and Bard, these LLMs manifested significantly elevated scores on the DIF and the DDF scales, indicating a notable deficiency in their ability to simulate the subjective human experience accurately. This indicates an inherent limitation in its capacity to convincingly replicate the nuances of human emotional introspection, specifically relating to individual concealment (DIF) and social inhibition (DDF) factors (Kmieciak, 2022). Research in cognitive psychology has corroborated that escalating DIF and DDF scores are typically associated with increased symptoms of depression and anxiety in humans (Shin, Yun, and Lee, 2022). Assuming that these psychometric parameters equally apply to LLMs such as GPT 3.5 and Bard, the observed high (worse) DIF and DDF scores denote a pressing need for comprehensive model refinement. Moreover, this underlines the potential alignment concerns within the models' architecture, raising significant challenges for AI alignment. As the technology community continues to scale large language models with synthetic data generated by predecessor LLMs, the propensity to exaggerate specific personality traits will increase by many folds. Experts acknowledge that there are risks with scaling large language models even further (Bender et al., 2021) and that there lies responsibility within the technical community to develop products with a process that mitigates harmful consequence and promote human well-being by adopting responsible computing practices (McMillan-Major, Bender, \u0026amp; Friedman, 2023).\u003c/p\u003e \u003cp\u003eIn conclusion, our exploration of LLMs such as Bard, GPT 3.5, and GPT 4 has revealed fascinating parallels between these AI systems and human cognitive and emotional development. Despite their lack of physical embodiment, these models demonstrate a form of simulated empathy and emotional understanding, challenging traditional assertions about the prerequisites for emotional intelligence (Pfeifer \u0026amp; Bongard, 2006; Goren et al., 2011). Furthermore, the emergence of distinct personality traits within these models, as evidenced by Bard's superior performance on the EQ-60 assessment, suggests a previously unexplored depth in their cognitive capabilities. The structure of Large Language Models (LLMs), such as GPT-3 developed by OpenAI, parallels human cognitive development (Brown et al., 2020). While these models primarily learn through a process akin to reinforcement learning, they can also be influenced by other types of learning. Much of their functionality is deeply rooted in their developmental connections, similar to the cognitive schemas in human psychology that evolve over time (Piaget, 1952).\u003c/p\u003e \u003cp\u003eWhile adjusting the weights and enhancing the model's proficiency with new data necessitates retraining. This process is analogous to the development of human intelligence, which begins with early neural wiring, pruning, and plasticity, irreversible processes (Huttenlocher \u0026amp; Dabholkar, 1997). Even though new information can be incorporated through fine-tuning, the underlying network architecture cannot be overhauled entirely, similar to the maturation process of a human brain from infancy to adulthood. It is plausible that this principle will also apply to AI models, suggesting that their fundamental structure, once established, cannot be entirely restructured (Bengio et al., 2013). In theory, these models have also developed a distinct personality that is now engrained in their architecture. While the capacity for \"steerability\" in these models allows them to simulate a diverse range of personas that mirrors human abilities, the manifestation of these traits raises critical questions about the spectrum of personality traits in LLMs and their alignment with socially accepted citizen agency promoting values (Coeckelbergh, 2023b).\u003c/p\u003e \u003cp\u003eThe performance of these models on assessments such as the TAS-20 and EQ-60 provides tangible metrics for assessing these personality traits and their alignment or misalignment with human benchmarks (Ngo et al., 2023). These results underscore the need for further research into the cognitive models underlying these LLMs, particularly given their underperformance in tasks associated with human working memory and emotion regulation (Jakobson, 2021). As AI systems continue to evolve, massive efforts and investments are being made to imbue them with elements of empathy and emotional intelligence, to achieve a true human-like artificial general intelligence (AGI). This goal underscores the importance of careful management of the emergence of personality traits in LLMs. As we continue to push the boundaries of AI capabilities, we must remain vigilant in our pursuit of value alignment, ensuring that these systems not only understand and emulate human emotions and values but also interact with us in a manner that is both empathetic, non-deceptive, and meet key requirements (D\u0026iacute;az-Rodr\u0026iacute;guez et al., 2023).\u003c/p\u003e \u003cp\u003eThe datasets generated during and/or analysed during the current study are available in the Figshare repository, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://figshare.com/articles/dataset/Identification_and_Description_of_Emotions_by_Current_Large_Language_Models_-_Dataset/24955719\u003c/span\u003e\u003cspan address=\"https://figshare.com/articles/dataset/Identification_and_Description_of_Emotions_by_Current_Large_Language_Models_-_Dataset/24955719\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eJ.F conceived the study. S.P performed the data collection and analysed the data. S.P wrote the manuscript. J.F. contributed to resources, review, editing, and supervision. All authors reviewed the manuscript.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eAaron, R.V., Fisher, E., de la Vega, R., Lumley, M.A., \u0026amp; Palermo, T.M. (2019). Alexithymia in individuals with chronic pain and its relation to pain intensity, physical interference, depression, and anxiety: a systematic review and meta-analysis. PAIN, pp. 160, 994\u0026ndash;1006.\u003c/li\u003e\n \u003cli\u003eAgbavor, Felix; Liang, Hualou (December 22, 2022). \u0026ldquo;Predicting dementia from spontaneous speech using large language models\u0026rdquo;. PLOS Digital Health. 1 (12): e0000168.\u003c/li\u003e\n \u003cli\u003eAstington, J.W., Harris, P.L., \u0026amp; Olson, D.R. (1988). Developing theories of mind.\u003c/li\u003e\n \u003cli\u003eBagby, R. M., Parker, J. D., \u0026amp; Taylor, G. J. (1994). The twenty-item Toronto Alexithymia Scale\u0026mdash;I. Item selection and cross-validation of the factor structure. Journal of psychosomatic research, 38(1), 23-32.\u003c/li\u003e\n \u003cli\u003eBaron-Cohen, S., \u0026amp; Wheelwright, S. (2004). The Empathy Quotient: An Investigation of Adults with Asperger Syndrome or High Functioning Autism, and Normal Sex Differences. Journal of Autism and Developmental Disorders, 34(2), 163-175.\u003c/li\u003e\n \u003cli\u003eBatson C. D. (2009). \u0026ldquo;These things called empathy: eight related but distinct phenomena,\u0026rdquo; in The Social Neuroscience of Empathy, eds Decety J., Ickes W. (Cambridge, MA: MIT Press; ), 3\u0026ndash;15. [Google Scholar] [Ref list]\u003c/li\u003e\n \u003cli\u003eBender, E. M., Gebru, T., McMillan-Major, A., \u0026amp; Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? . In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT \u0026apos;21) (pp. 610\u0026ndash;623). Association for Computing Machinery. https://doi.org/10.1145/3442188.3445922\u003c/li\u003e\n \u003cli\u003eBengio, Y., Courville, A., \u0026amp; Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8), 1798-1828.\u003c/li\u003e\n \u003cli\u003eBerthoz, S., Wessa, M., K\u0026eacute;dia, G., Wicker, B., \u0026amp; Gr\u0026egrave;zes, J. (2008). Cross-Cultural Validation of the Empathy Quotient in a French-Speaking Sample. The Canadian Journal of Psychiatry, 53, 469 - 477.\u003c/li\u003e\n \u003cli\u003eBostrom, N. (2020). Ethical Issues in Advanced Artificial Intelligence.\u003c/li\u003e\n \u003cli\u003eBrown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T.J., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., \u0026amp; Amodei, D. (2020). Language Models are Few-Shot Learners. ArXiv, abs/2005.14165.\u003c/li\u003e\n \u003cli\u003eBryant, P.T. (2021). Cognitive Empathy. Augmented Humanity.\u003c/li\u003e\n \u003cli\u003eBubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., Nori, H., Palangi, H., Ribeiro, M. T., \u0026amp; Zhang, Y. (2023, March). Sparks of Artificial General Intelligence: Early experiments with GPT-4 [Unpublished manuscript]. Microsoft Research. Retrieved from https://www.microsoft.com/en-us/research/publication/sparks-of-artificial-general-intelligence-early-experiments-with-gpt-4/\u003c/li\u003e\n \u003cli\u003eBurnell, R., Schellaert, W., Burden, J., Ullman, T. D., Martinez-Plumed, F., Tenenbaum, J. B., Rutar, D., Cheke, L. G., Sohl-Dickstein, J., Mitchell, M., Kiela, D., Shanahan, M., Voorhees, E. M., Cohn, A. G., Leibo, J. Z., \u0026amp; Hernandez-Orallo, J. (2023). Rethink reporting of evaluation results in AI. Science, 380(6641), 136-138. https://doi.org/10.1126/science.adf6369\u003c/li\u003e\n \u003cli\u003eCoeckelbergh, M. (2023a). How to do robots with words: A performative view of the moral status of humans and nonhumans. Ethics and Information Technology, 25, 44. https://doi.org/10.1007/s10676-023-09719-5\u003c/li\u003e\n \u003cli\u003eCoeckelbergh, M. (2023b). Democracy, epistemic agency, and AI: Political epistemology in times of artificial intelligence. AI Ethics, 3, 1341-1350. https://doi.org/10.1007/s43681-022-00239-4\u003c/li\u003e\n \u003cli\u003eCui, Z., \u0026amp; Liu, J. (2022). A Study on Two Conditions for the Realization of Artificial Empathy and Its Cognitive Foundation. Philosophies.\u003c/li\u003e\n \u003cli\u003eChollet, F. (2019). On the Measure of Intelligence. arXiv preprint arXiv:1911.01547.\u003c/li\u003e\n \u003cli\u003eCzerwinski, M. (Host). (2018, April 18). 020 Getting good VIBEs from your computer with Dr. Mary Czerwinski [Audio podcast episode]. In Getting good VIBEs from your computer with Dr. Mary Czerwinski. Microsoft Research Podcast. Retrieved from https://www.microsoft.com/en-us/research/podcast/getting-good-vibes-from-your-computer-with-dr-mary-czerwinski/\u003c/li\u003e\n \u003cli\u003eDaniel, M., Peter, L., \u0026amp; Bileviciute-Ljungar, I. (2018). The Relationship Between Alexithymia and Emotional Awareness: A Meta-Analytic Review of the Correlation Between TAS-20 and LEAS. Frontiers in Psychology, 9, 453. https://doi.org/10.3389/fpsyg.2018.00453\u003c/li\u003e\n \u003cli\u003eDataCamp. (2023, March). What is GPT-4 and Why Does it Matter? Retrieved from https://www.datacamp.com/blog/what-we-know-gpt4\u003c/li\u003e\n \u003cli\u003eDavis, M.H., Davis, M.P., Davis, M., Davis, M., Davis, M., Davis, M., Davis, M., Davis, F.C., Davis, H.A., \u0026amp; Davis, I.W. (1980). A Multidimensional Approach to Individual Differences in Empathy.\u003c/li\u003e\n \u003cli\u003eDemers, L.A., \u0026amp; Koven, N.S. (2015). The relation of alexithymic traits to affective theory of mind. The American Journal of Psychology, 128(1), 31-42.\u003c/li\u003e\n \u003cli\u003eDe Waal F. B. M. (2008). Putting the altruism back into altruism: the evolution of empathy. Ann. Rev. Psychol. 59 279\u0026ndash;300. 10.1146/annurev.psych.59.103006.093625\u003c/li\u003e\n \u003cli\u003eDhingra, S., Singh, M., VaisakhS., B., Malviya, N., \u0026amp; Gill, S.S. (2023). Mind meets machine: Unravelling GPT-4\u0026apos;s cognitive psychology. ArXiv, abs/2303.11436.\u003c/li\u003e\n \u003cli\u003eDillion, D., Tandon, N., Gu, Y., \u0026amp; Gray, K. (2023). Can AI language models replace human participants? Trends in Cognitive Sciences, 27(7), 597\u0026ndash;600. https://doi.org/10.1016/j.tics.2023.04.008\u003c/li\u003e\n \u003cli\u003eDreyfus, H. (1992). What Computers Still Can\u0026apos;t Do: A Critique of Artificial Reason. MIT Press.\u003c/li\u003e\n \u003cli\u003eEisenberg N., Strayer J. (1987). Empathy and Its Development. New York, NY: Cambridge University Press. [Google Scholar] [Ref list]\u003c/li\u003e\n \u003cli\u003eFloreano, D., D\u0026uuml;rr, P., \u0026amp; Mattiussi, C. (2008). Evolution of embodied intelligence. In Embodied artificial intelligence (pp. 293-311). Springer.\u003c/li\u003e\n \u003cli\u003eFriston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience, 11(2), 127\u0026ndash;138.\u003c/li\u003e\n \u003cli\u003eGallese, V., \u0026amp; Goldman, A.I. (1998). Mirror neurons and the simulation theory of mind-reading. \u003cem\u003eTrends in Cognitive Sciences, 2\u003c/em\u003e, 493-501.\u003c/li\u003e\n \u003cli\u003eGladstein G. A. (1983). Understanding empathy: integrating counseling, developmental, and social psychology perspectives. J. Counsel. Psychol. 30 467\u0026ndash;482. 10.1037/0022-0167.30.4.467\u003c/li\u003e\n \u003cli\u003eGoodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A.C., \u0026amp; Bengio, Y. (2014). Generative Adversarial Nets. NIPS.\u003c/li\u003e\n \u003cli\u003eGoren, G., Kaplan, D., Lankow, B., Little, D., Sherwin, J., Suter, B., \u0026amp; Thaler, L. (2011). Toward an Integrated Approach to Perception and Action: Conference Report and Future Directions. Frontiers in Systems Neuroscience, 5. https://doi.org/10.3389/fnsys.2011.00020\u003c/li\u003e\n \u003cli\u003eGrynberg, D., Luminet, O., Corneille, O., Gr\u0026egrave;zes, J., \u0026amp; Berthoz, S. (2010). Alexithymia in the interpersonal domain: A general deficit of empathy?. Personality and Individual Differences, 49(8), 845-850. https://doi.org/10.1016/j.paid.2010.07.013\u003c/li\u003e\n \u003cli\u003eHassabis, D., Spreng, R. N., Rusu, A. A., Robbins, C. A., Mar, R. A., \u0026amp; Schacter, D. L. (2014). Imagine all the people: How the brain creates and uses personality models to predict behavior. Cerebral Cortex, 24(8), 1979-1987.\u003c/li\u003e\n \u003cli\u003eHagendorff, Thilo (2023): Deception Abilities Emerged in Large Language Models. In arXiv:2307.16513, pp. 1\u0026ndash;21.\u003c/li\u003e\n \u003cli\u003eHoffman M. L. (2000). Empathy and Moral Development: Implications for Caring and Justice. Cambridge: Cambridge University Press, 10.1017/CBO9780511805851\u003c/li\u003e\n \u003cli\u003eHuttenlocher, P. R., \u0026amp; Dabholkar, A. S. (1997). Regional differences in synaptogenesis in human cerebral cortex. Journal of Comparative Neurology, 387(2), 167-178.\u003c/li\u003e\n \u003cli\u003eJakobson, L.S., \u0026amp; Rigby, S.N. (2021). Alexithymia and Sensory Processing Sensitivity: Areas of Overlap and Links to Sensory Processing Styles. Frontiers in Psychology, 12.\u003c/li\u003e\n \u003cli\u003eJiang, H., Zhang, X., Cao, X., Kabbara, J., \u0026amp; Roy, D. (2023). PersonaLLM: Investigating the Ability of GPT-3.5 to Express Personality Traits and Gender Differences. ArXiv, abs/2305.02547.\u003c/li\u003e\n \u003cli\u003eKılı\u0026ccedil;, F., Demirdaş, A., Işık, \u0026Uuml;., Akkuş, M., Atay, I.M., \u0026amp; Kuzug\u0026uuml;denlioġlu, D. (2020). Empathy, Alexithymia, and Theory of Mind in Borderline Personality Disorder. Journal of Nervous \u0026amp; Mental Disease.\u003c/li\u003e\n \u003cli\u003eKim E. K., You S., Knox J. (2020). The mediating effect of empathy on the relation between child self-expressiveness in family and prosocial behaviors. J. Child Fam. Stud. 29 1572\u0026ndash;1581. 10.1007/s10826-019-01676-2 [CrossRef] [Google Scholar] [Ref list]\u003c/li\u003e\n \u003cli\u003eKmieciak, R. (2022). Alexithymia, social inhibition, affectivity, and knowledge hiding. J. Knowl. Manag., pp. 26, 461\u0026ndash;485.\u003c/li\u003e\n \u003cli\u003eKojima, M., Frasure-Smith, N., \u0026amp; Lesp\u0026eacute;rance, F. (2001). Alexithymia following myocardial infarction: Psychometric properties and correlates of the Toronto Alexithymia Scale. Journal of Psychosomatic Research, 51(3), 487-495. https://doi.org/10.1016/S0022-3999(01)00253-7\u003c/li\u003e\n \u003cli\u003eKurzweil, R. (n.d.). By 2029, computers will have emotional intelligence and be convincing as people.\u003c/li\u003e\n \u003cli\u003eLane, R. D., Sechrest, L., Reidel, R., Weldon, V., Kaszniak, A., and Schwartz, G. E. (1996). Impaired verbal and nonverbal emotion recognition in alexithymia. Psychosom. Med. 58, 203\u0026ndash;210. doi: 10.1097/00006842-199605000-00002\u003c/li\u003e\n \u003cli\u003eLawrence, E. J., Shaw, P., Baker, D., Baron-Cohen, S., \u0026amp; David, A. S. (2004). Measuring empathy:\u003c/li\u003e\n \u003cli\u003eLeDoux, J. E. (1996). The emotional brain: The mysterious underpinnings of emotional life. New York, NY: Simon \u0026amp; Schuster.\u003c/li\u003e\n \u003cli\u003eLee, C.-C., Chaspari, T., Provost, E. M., \u0026amp; Narayanan, S. S. (2023). An Engineering View on Emotions and Speech: From Analysis and Predictive Models to Responsible Human-Centered Applications. Proceedings of the IEEE, 111(10), 1142-1158. https://doi.org/10.1109/JPROC.2023.3276209\u003c/li\u003e\n \u003cli\u003eLiu, H., Ning, R., Teng, Z., Liu, J., Zhou, Q., \u0026amp; Zhang, Y. (2023). Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4. ArXiv, abs/2304.03439.\u003c/li\u003e\n \u003cli\u003eMart\u0026iacute;nez, E. (2023). Re-Evaluating GPT-4\u0026apos;s Bar Exam Performance. SSRN Electronic Journal.\u003c/li\u003e\n \u003cli\u003eMartino, G., Caputo, A., Vicario, C.M., Catalano, A., Schwarz, P., \u0026amp; Quattropani, M.C. (2020). The Relationship Between Alexithymia and Type 2 Diabetes: A Systematic Review. Frontiers in Psychology, 11.\u003c/li\u003e\n \u003cli\u003eMasten C. L., Morelli S. A., Eisenberger N. I. (2011). An fMRI investigation of empathy for \u0026lsquo;social pain\u0026rsquo; and subsequent prosocial behavior. Neuroimage 55 381\u0026ndash;388. 10.1016/j.neuroimage.2010.11.060 [PubMed] [CrossRef] [Google Scholar] [Ref list]\u003c/li\u003e\n \u003cli\u003eMcCulloch, W. S., \u0026amp; Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5(4), 115\u0026ndash;133.\u003c/li\u003e\n \u003cli\u003eMcMillan-Major, A., Bender, E. M., \u0026amp; Friedman, B. (2023). Data Statements: From Technical Concept to Community Practice. ACM Journal of Responsible Computing. Just Accepted. https://doi.org/10.1145/3594737\u003c/li\u003e\n \u003cli\u003eMeđedović, J., \u0026amp; Đuričić, N. (2018). Delineating Psychopathy from Cognitive Empathy. European journal of analytic philosophy.\u003c/li\u003e\n \u003cli\u003eMicrosoft Research. (n.d.). HUE: Human Understanding and Empathy. Microsoft. Retrieved July 3, 2023, from https://www.microsoft.com/en-us/research/group/human-understanding-and-empathy/\u003c/li\u003e\n \u003cli\u003eMinsky, M. (2006). The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind. New York, NY: Simon \u0026amp; Schuster.\u003c/li\u003e\n \u003cli\u003eMitchell, M., \u0026amp; Krakauer, D. C. (2022). The Debate Over Understanding in AI\u0026apos;s Large Language Models. ArXiv. https://doi.org/10.1073/pnas.2215907120\u003c/li\u003e\n \u003cli\u003eMyers, I.B. (1962). The myers-briggs type indicator.\u003c/li\u003e\n \u003cli\u003eNemiah, J. C., Freyberger, H., Sifneos, P. E., \u0026amp; Hill, O. W. (1976). Modern trends in psychosomatic medicine. Alexithymia: a view of the psychosomatic process, 3, 430\u0026ndash;9.\u003c/li\u003e\n \u003cli\u003eNgo, R. (2022). The alignment problem from a deep learning perspective. ArXiv, abs/2209.00626.\u003c/li\u003e\n \u003cli\u003eNori, H., King, N., McKinney, S.M., Carignan, D., \u0026amp; Horvitz, E. (2023). Capabilities of GPT-4 on Medical Challenge Problems. ArXiv, abs/2303.13375.\u003c/li\u003e\n \u003cli\u003eOmdahl B. L. (1995). Cognitive Appraisal, Emotion, and Empathy. Mahwah, NJ: Lawrence Erlbaum.\u003c/li\u003e\n \u003cli\u003eOpenAI. (2023, April 5). Our approach to AI safety. Retrieved from https://openai.com/blog/our-approach-to-ai-safety#OpenAI\u003c/li\u003e\n \u003cli\u003eOpenAI (2023b). GPT-4 Technical Report. ArXiv, abs/2303.08774.\u003c/li\u003e\n \u003cli\u003eOpenAI. (2022, November 30). Introducing ChatGPT. OpenAI. https://openai.com/blog/chatgpt\u003c/li\u003e\n \u003cli\u003eParker JD, Taylor GJ, Bagby RM (2003) The 20-Item Toronto Alexithymia\u003c/li\u003e\n \u003cli\u003eParker, J. D. A., Taylor, G. J., \u0026amp; Bagby, R. M. (2001). The relationship between emotional intelligence and alexithymia. Personality and Individual Differences, 30(1), 107-115. https://doi.org/10.1016/S0191-8869(00)00014-3\u003c/li\u003e\n \u003cli\u003eParker, J. D., Taylor, G. J., \u0026amp; Bagby, R. M. (2003). The 20-Item Toronto Alexithymia Scale. III. Reliability and factorial validity in a community population. Journal of Psychosomatic Research, 55(3), 269-275. doi: 10.1016/s0022-3999(02)00578-0.\u003c/li\u003e\n \u003cli\u003ePaunonen, S.V., \u0026amp; Ashton, M.C. (2001). Big five factors and facets and the prediction of behavior. Journal of Personality and social psychology, p. 81 3, 524\u0026ndash;39.\u003c/li\u003e\n \u003cli\u003ePaunonen, S.V., Haddock, G., Forsterling, F., \u0026amp; Keinonen, M. (2003). Broad versus narrow personality measures and the prediction of behaviour across cultures. European Journal of Personality, 17, 413 - 433.\u003c/li\u003e\n \u003cli\u003ePfeifer, R. (2000). On the role of embodiment in the emergence of cognition and emotion. In H. Hatano, N. Okada, \u0026amp; H. Tanabe (Eds.), Affective minds (pp. 43-57). Amsterdam: Elsevier.\u003c/li\u003e\n \u003cli\u003ePfeifer, R., \u0026amp; Bongard, J. (2006). How the body shapes the way we think: A new view of intelligence. MIT Press.\u003c/li\u003e\n \u003cli\u003ePham, T., Ducro, C., \u0026amp; Luminet, O. (2010). Psychopathy, Alexithymia and Emotional Intelligence in a Forensic Hospital. International Journal of Forensic Mental Health, 9, 24 - 32.\u003c/li\u003e\n \u003cli\u003ePiaget, J. (1952). The origins of intelligence in children (Vol. 8, No. 5, pp. 18-1952). New York: International Universities Press.\u003c/li\u003e\n \u003cli\u003ePremack, D., \u0026amp; Woodruff, G. (1978). Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences, pp. 1, 515 \u0026ndash; 526.\u003c/li\u003e\n \u003cli\u003ePreti, A., Vellante, M., Baron-Cohen, S., Zucca, G., Petretto, D.R., \u0026amp; Masala, C. (2011). The Empathy Quotient: A cross-cultural comparison of the Italian version. Cognitive Neuropsychiatry, 16, 50 - 70.\u003c/li\u003e\n \u003cli\u003eRahimi Moghaddam, S., \u0026amp; Honey, C.J. (2023). Boosting Theory-of-Mind Performance in Large Language Models via Prompting. ArXiv, abs/2304.11490.\u003c/li\u003e\n \u003cli\u003eReliability and validity of the Empathy Quotient. Psychological Medicine, 34, 911919.\u003c/li\u003e\n \u003cli\u003eRosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386. [Link]\u003c/li\u003e\n \u003cli\u003eRumelhart, D. E., Hinton, G. E., \u0026amp; Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533-536. [Link]\u003c/li\u003e\n \u003cli\u003eSearle, J. R. (1980). Minds, brains, and programs. *Behavioral and Brain Sciences*, *3*(3), 417-457.\u003c/li\u003e\n \u003cli\u003eSchimmenti, A., Jonason, P.K., Passanisi, A., La Marca, L., Di Dio, N., \u0026amp; Gervasi, A.M. (2017). Exploring the Dark Side of Personality: Emotional Awareness, Empathy, and the Dark Triad Traits in an Italian Sample. Current Psychology, pp. 38, 100\u0026ndash;109.\u003c/li\u003e\n \u003cli\u003eSchmukle, S.C., Sch\u0026uuml;tz, A., Sękowski, A.E., \u0026amp; Stahl, J. (2015). A Multidimensional View of the Relationship Between Empathy and the Dark Triad.\u003c/li\u003e\n \u003cli\u003eShin, J., Yun, S.J., \u0026amp; Lee, T.K. (2022). Identification and Characterization of Alexithymia Subgroups by Latent Profile Analysis of TAS-20K. STRESS.\u003c/li\u003e\n \u003cli\u003eSinger, T. (2006). The neuronal basis and ontogeny of empathy and mind reading: Review of literature and implications for future research. Neuroscience \u0026amp; Biobehavioral Reviews, pp. 30, 855\u0026ndash;863.\u003c/li\u003e\n \u003cli\u003eSuttie, J. (2014, February 4). Can a psychopath learn to feel pain? Greater Good Magazine, Retrieved from https://greatergood.berkeley.edu/article/item/can_a_psychopath_learn_feel_pain\u003c/li\u003e\n \u003cli\u003eTaylor, G., Bagby, R., and Parker, J. (1997). Disorders of Affect Regulation. Alexthymia in Medical and Psychiatric Illness. Cambridge: University Press.\u003c/li\u003e\n \u003cli\u003eWang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., \u0026amp; Zhang, W. (2022). A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances. ArXiv, abs/2203.06935.\u003c/li\u003e\n \u003cli\u003eWortman, B.J., \u0026amp; Wang, J.Z. (2022). HICEM: A High-Coverage Emotion Model for Artificial Emotional Intelligence. ArXiv, abs/2206.07593.\u003c/li\u003e\n \u003cli\u003eWu, J. (2019, December 17). Empathy in Artificial Intelligence. Forbes. Retrieved from https://www.forbes.com/sites/cognitiveworld/2019/12/17/empathy-in-artificial-intelligence/?sh=6a4fa1b46327\u003c/li\u003e\n \u003cli\u003eYal\u0026ccedil;ın, \u0026Ouml;. N., \u0026amp; DiPaola, S. (2020). Modeling empathy: Building a link between affective and cognitive processes. Artificial Intelligence Review, 53, 2983\u0026ndash;3006. https://doi.org/10.1007/s10462-019-09753-0\u003c/li\u003e\n \u003cli\u003eZaveria, Z. (2023, April 7). What is Artificial Emotional Intelligence? Retrieved from https://www.analyticsinsight.net/what-is-artificial-emotional-intelligence/\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Alexithymia, Empathy, Large Language Models, Emotional Intelligence, Alignment Problem","lastPublishedDoi":"10.21203/rs.3.rs-4146250/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4146250/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThe assertion that artificial intelligence (AI) cannot grasp the complexities of human emotions has been a long-standing debate. However, recent advancements in large language models (LLMs) challenge this notion by demonstrating an increased capacity for understanding and generating human-like text. In this study, we evaluated the empathy levels and the identification and description of emotions by three current language models: Bard, GPT 3.5, and GPT 4. We used the Toronto Alexithymia Scale (TAS-20) and the 60-question Empathy Quotient (EQ-60) questions to prompt these models and score the responses. The models' performance was contrasted with human benchmarks of neurotypical controls and clinical populations. We found that the less sophisticated models (Bard and GPT 3.5) performed inferiorly on TAS-20, aligning close to alexithymia, a condition with significant difficulties in recognizing, expressing, and describing one's or others' experienced emotions. However, GPT 4 achieved performance close to the human level. These results demonstrated that LLMs are comparable in their ability to identify and describe emotions and may be able to surpass humans in their capacity for emotional intelligence. Our novel insights provide alignment research benchmarks and a methodology for aligning AI with human values, leading toward an empathetic AI that mitigates risk.\u003c/p\u003e","manuscriptTitle":"Identification and Description of Emotions by Current Large Language Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-04-01 18:29:00","doi":"10.21203/rs.3.rs-4146250/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"cce1560b-059c-447d-b12b-e212b28cd633","owner":[],"postedDate":"April 1st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":30064389,"name":"Biological sciences/Computational biology and bioinformatics/Computational neuroscience/Learning algorithms"},{"id":30064390,"name":"Biological sciences/Psychology/Human behaviour"},{"id":30064391,"name":"Physical sciences/Mathematics and computing/Computational science"},{"id":30064392,"name":"Biological sciences/Neuroscience/Cognitive neuroscience/Intelligence"},{"id":30064393,"name":"Biological sciences/Neuroscience/Cognitive neuroscience/Language"},{"id":30064394,"name":"Biological sciences/Neuroscience/Cognitive neuroscience/Personality"},{"id":30064395,"name":"Health sciences/Neurology/Neurological disorders/Neurodevelopmental disorders/Autism spectrum disorders"}],"tags":[],"updatedAt":"2024-08-29T04:48:44+00:00","versionOfRecord":[],"versionCreatedAt":"2024-04-01 18:29:00","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4146250","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4146250","identity":"rs-4146250","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}