Multimodal large language models converge on the human-like geometry of abstract emotion

Multimodal large language models converge on the human-like geometry of abstract emotion | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Multimodal large language models converge on the human-like geometry of abstract emotion Huiguang He, Changde Du, Yizhuo Lu, Zhongyu Huang, Yi Sun, Zisen Zhou, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8859558/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted You are reading this latest preprint version Abstract Understanding whether artificial intelligence (AI) systems represent abstract concepts in a human-like manner is pivotal for developing trustworthy AI. While recent work has aligned model representations with human concrete visual object concepts, it remains unclear whether such alignment extends to the subjective and context-dependent domain of emotion. Here, we investigate the emergent affective geometry in large language models (LLMs) and multimodal LLMs (MLLMs) through a large-scale, unsupervised machine-behavioral'' paradigm. By deriving 30-dimensional embeddings from over 12 million triplet odd-one-out judgments on 2,180 emotionally evocative videos, we reveal a sophisticated hybrid'' geometry. This structure synthesizes categorical clusters with continuous dimensions, showing strong selective correlations with human ratings across 34 emotion categories and 14 affective dimensions, effectively reconciling the long-standing category-versus-dimension debate in affective science. To demonstrate the operational utility of these representations, we introduce a generative editing framework, showing that manipulating specific affective components actively steers generated video content in a predictable, human-interpretable manner. Crucially, at the neural level, the MLLM-derived affective space predicts human fMRI activity in high-level social-emotional regions (e.g., temporoparietal junction) with accuracy matching or exceeding traditional human self-report ratings. These findings demonstrate that MLLMs converge on a biologically plausible, brain-aligned representational scheme for abstract emotion, distinguishing them from models of pure visual perception and establishing a framework for artificial social intelligence. Biological sciences/Neuroscience/Computational neuroscience/Neural encoding Biological sciences/Neuroscience/Emotion Biological sciences/Neuroscience/Cognitive neuroscience/Intelligence Physical sciences/Mathematics and computing/Computer science Biological sciences/Psychology/Human behaviour Full Text Additional Declarations There is NO Competing Interest. Cite Share Download PDF Status: Under Review Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8859558","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":590149733,"identity":"fbe04a0a-4d4c-4388-a8f9-13e25b96deb2","order_by":0,"name":"Huiguang He","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA4klEQVRIiWNgGAWjYBACAwbGBghLgoHxQQKpWpgNiNQCAxIMbBJEOcxcIrn5NU/NHbsG6eZjFQ9qGOT5G5ifPcCnxXJGYps1z7FnyQ0yx9JuJBxjMJxxgM3cAJ8WgxuJbcY8bIeTGSRyzG4ksDEwbmDgwe9CiJZ/IC353woS/jHYE6Ol+TFv22E7oC1sDIltDImEtZx52MY4t+9wAptEmrFEYp9E8ozDbGb4tRxPf/zhzbfD9vwSyQ8//vhmY9vf3vyMUGiDnQF0EhgA2cwE1AMB8wcgYU9Y3SgYBaNgFIxYAABodEXV0LBXeAAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0002-0684-1711","institution":"Institute of Automation, Chinese Academy of Sciences","correspondingAuthor":true,"prefix":"","firstName":"Huiguang","middleName":"","lastName":"He","suffix":""},{"id":590149734,"identity":"60aff1e4-c69e-4af0-8b60-56f6e7c3c47d","order_by":1,"name":"Changde Du","email":"","orcid":"https://orcid.org/0000-0002-0084-433X","institution":"Institute of Automation, Chinese Academy of Sciences","correspondingAuthor":false,"prefix":"","firstName":"Changde","middleName":"","lastName":"Du","suffix":""},{"id":590149735,"identity":"67993f76-5cf9-49e5-9e5e-9dc484f0f2df","order_by":2,"name":"Yizhuo Lu","email":"","orcid":"","institution":"Institute of Automation, Chinese Academy of Sciences","correspondingAuthor":false,"prefix":"","firstName":"Yizhuo","middleName":"","lastName":"Lu","suffix":""},{"id":590149736,"identity":"72e82246-08c4-4538-8209-19c27856f7f6","order_by":3,"name":"Zhongyu Huang","email":"","orcid":"","institution":"Institute of Automation, Chinese Academy of Sciences","correspondingAuthor":false,"prefix":"","firstName":"Zhongyu","middleName":"","lastName":"Huang","suffix":""},{"id":590149737,"identity":"6e6554d0-e1cd-4c41-a717-8cb2a11c4736","order_by":4,"name":"Yi Sun","email":"","orcid":"","institution":"Institute of Automation, Chinese Academy of Sciences","correspondingAuthor":false,"prefix":"","firstName":"Yi","middleName":"","lastName":"Sun","suffix":""},{"id":590149738,"identity":"9f6a5edd-5361-4fc8-a85f-dfe0c200bf3f","order_by":5,"name":"Zisen Zhou","email":"","orcid":"","institution":"Beijing Normal University","correspondingAuthor":false,"prefix":"","firstName":"Zisen","middleName":"","lastName":"Zhou","suffix":""},{"id":590149739,"identity":"1b8720ff-dccc-4e5b-9892-622509b5eca7","order_by":6,"name":"Shaozheng Qin","email":"","orcid":"","institution":"Beijing Normal University","correspondingAuthor":false,"prefix":"","firstName":"Shaozheng","middleName":"","lastName":"Qin","suffix":""}],"badges":[],"createdAt":"2026-02-12 08:30:25","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8859558/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8859558/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":106093655,"identity":"dc4b8a77-7ebc-4f53-be36-655afe519e8d","added_by":"auto","created_at":"2026-04-03 11:38:27","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":30389815,"visible":true,"origin":"","legend":"Article File","description":"","filename":"main8NMI.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8859558/v1_covered_48447c3a-b0cb-4435-8a18-2e5bb3145c74.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Multimodal large language models converge on the human-like geometry of abstract emotion","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-8859558/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8859558/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Understanding whether artificial intelligence (AI) systems represent abstract concepts in a human-like manner is pivotal for developing trustworthy AI. While recent work has aligned model representations with human concrete visual object concepts, it remains unclear whether such alignment extends to the subjective and context-dependent domain of emotion. Here, we investigate the emergent affective geometry in large language models (LLMs) and multimodal LLMs (MLLMs) through a large-scale, unsupervised ``machine-behavioral'' paradigm. By deriving 30-dimensional embeddings from over 12 million triplet odd-one-out judgments on 2,180 emotionally evocative videos, we reveal a sophisticated ``hybrid'' geometry. This structure synthesizes categorical clusters with continuous dimensions, showing strong selective correlations with human ratings across 34 emotion categories and 14 affective dimensions, effectively reconciling the long-standing category-versus-dimension debate in affective science. To demonstrate the operational utility of these representations, we introduce a generative editing framework, showing that manipulating specific affective components actively steers generated video content in a predictable, human-interpretable manner. Crucially, at the neural level, the MLLM-derived affective space predicts human fMRI activity in high-level social-emotional regions (e.g., temporoparietal junction) with accuracy matching or exceeding traditional human self-report ratings. These findings demonstrate that MLLMs converge on a biologically plausible, brain-aligned representational scheme for abstract emotion, distinguishing them from models of pure visual perception and establishing a framework for artificial social intelligence.","manuscriptTitle":"Multimodal large language models converge on the human-like geometry of abstract emotion","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-02 02:29:15","doi":"10.21203/rs.3.rs-8859558/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-machine-intelligence","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"natmachintell","sideBox":"Learn more about [Nature Machine Intelligence](http://www.nature.com/natmachintell/)","snPcode":"","submissionUrl":"","title":"Nature Machine Intelligence","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Research","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"906dc785-feef-43d3-9c25-8d5320deba45","owner":[],"postedDate":"April 2nd, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":62794790,"name":"Biological sciences/Neuroscience/Computational neuroscience/Neural encoding"},{"id":62794791,"name":"Biological sciences/Neuroscience/Emotion"},{"id":62794792,"name":"Biological sciences/Neuroscience/Cognitive neuroscience/Intelligence"},{"id":62794793,"name":"Physical sciences/Mathematics and computing/Computer science"},{"id":62794794,"name":"Biological sciences/Psychology/Human behaviour"}],"tags":[],"updatedAt":"2026-04-22T13:05:28+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-02 02:29:15","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8859558","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8859558","identity":"rs-8859558","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}