Audio-Visual-Textual Fusion for Big Five Personality Prediction Using Deep Feature Extraction and Multimodal Transformers

Audio-Visual-Textual Fusion for Big Five Personality Prediction Using Deep Feature Extraction and Multimodal Transformers | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 3 February 2026 V1 Latest version Share on Audio-Visual-Textual Fusion for Big Five Personality Prediction Using Deep Feature Extraction and Multimodal Transformers Authors : Mubashra Fayyaz 0009-0008-4488-9768 [email protected] , Rauf Ahmed Shams Malick , and Adnan Akhunzada Authors Info & Affiliations https://doi.org/10.22541/au.177012929.90537214/v1 175 views 55 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract This study presents a data-efficient, interpretable framework for personality trait prediction using multimodal behavioral cues. Leveraging both handcrafted and deep features from video, audio, and text, and integrating them through a Multimodal Transformer with cross-modal attention, our approach captures rich intra- and inter-modal dependencies critical to accurate personality inference. To enhance interpretability, we introduced a Gaussian Mixture Model (GMM)-based post-processing step that transforms continuous regression outputs into discrete personality categories—Low, Medium, and High. This conversion enables intuitive and human-readable predictions, making the system suitable for real-world applications such as recruitment, adaptive education, and behavioral monitoring. Experimental results show that fusing modalities with feature diversity and cross-modal attention yields robust personality prediction performance even with limited training data. Supplementary Material File (audio_visual_textual_fusion_for_big_five_personality_prediction_using_deep_feature_extraction_and_multimodal_transformers.pdf) Download 4.81 MB Information & Authors Information Version history V1 Version 1 03 February 2026 Copyright This work is licensed under a Non Exclusive No Reuse License. Keywords cross-modal attention gaussian mixture modeling handcrafted features multimodal learning personality trait prediction Authors Affiliations Mubashra Fayyaz 0009-0008-4488-9768 [email protected] FAST National University of Computer and Emerging Sciences View all articles by this author Rauf Ahmed Shams Malick Ghazali University(GUTech View all articles by this author Adnan Akhunzada University of Doha for Science & Technology View all articles by this author Metrics & Citations Metrics Article Usage 175 views 55 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Mubashra Fayyaz, Rauf Ahmed Shams Malick, Adnan Akhunzada. Audio-Visual-Textual Fusion for Big Five Personality Prediction Using Deep Feature Extraction and Multimodal Transformers. Authorea . 03 February 2026. DOI: https://doi.org/10.22541/au.177012929.90537214/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.177012929.90537214/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9fe35c0aec24ad07',t:'MTc3OTE5NjQxMg=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();