Robust Audio-Image Steganography using Cross-Modal Based Transformer Models

Robust Audio-Image Steganography using Cross-Modal Based Transformer Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Robust Audio-Image Steganography using Cross-Modal Based Transformer Models Mark Taremwa, Roger Nick Anaedevha, Alexander Genadievich Trofimov This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5463235/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 7 You are reading this latest preprint version Abstract This research investigates the use of Vision Transformers (ViT), Audio Spectrogram Transformers (AST), and Cross-Modal Transformers (CMT) in audio-image fusion tasks, aiming to improve the representation learning and interaction between auditory and visual data. The ViT model extracts visual features from image patches resized to 224x224 pixels, while the AST model converts audio signals into mel spectrograms to capture detailed auditory features. The central focus is on the robust CMT model, which integrates visual and auditory features through a cross-modal attention mechanism. Extensive experiments using a diverse audio-image dataset from Kaggle reveal significant improvements. The initial developed ViT model enhances image embedding capacity by 12%, the AST model improves audio embedding tasks by 15%, and the CMT model achieves and overall embedding capacity increased by an average of 58.7%, an image PSNR improved by an average of 57.5%, image MSE reduced by an average of 87.3% as the CMT model shows more consistent performance across all the different stegos. this enhancement in cross-modal retrieval tasks, highlighting the effectiveness of robust CMT model in learning and utilizing inter-modal relationships. The proposed robust cross-modal attention mechanism outperforms traditional ViT model and Least Significant Bits [LSB] concatenation algorithms in feature alignment accuracy. Ablation studies further validate the robustness of this approach, demonstrating the contribution of each component to overall performance. This research establishes the viability and superiority of transformer-based architectures in audio-image fusion tasks, suggesting potential advancements in cross-modal tasks like audio-visual synchronization and multimodal sentiment analysis. Steganography Deep learning Vision Transformers Audio transformers Cross – Modal Transformers Audio-to-image Data hiding Secure communication Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviewers agreed at journal 17 Sep, 2025 Reviews received at journal 25 Jan, 2025 Reviewers agreed at journal 25 Dec, 2024 Reviewers invited by journal 25 Dec, 2024 Editor assigned by journal 16 Nov, 2024 Submission checks completed at journal 16 Nov, 2024 First submitted to journal 15 Nov, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5463235","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":390260872,"identity":"921d15f6-19f4-4b00-a21f-b158224413c1","order_by":0,"name":"Mark Taremwa","email":"","orcid":"","institution":"Southwest Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Mark","middleName":"","lastName":"Taremwa","suffix":""},{"id":390260873,"identity":"4f3b9d3b-15bc-438f-964e-07c425d6e213","order_by":1,"name":"Roger Nick Anaedevha","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABH0lEQVRIiWNgGAWjYPACCSAEATYGBgMG5gMgERlStLAlgER4iLAHroXHAMTEqcW8vf3i48odFvKS7WcPfvhQZpO4nf3M51c3aix4GNgPH92ARYvMmTPFhmfPSBjO5slLlpxxLi1xZ0/uNuucY0CH8aSl3cDmHomcNMnGNokEOYYcA2netsPGBgdytxnnsAG1SPCY4dCS/hOshf+N8e+/bf+NDc6/eWac8w+flvRjjCAt0hI5ZtKMbQfkDG7kMD/ObcOjhecMM8hhhjNnvDGz7DmXDNTyzIw5t0+Chw2XX9jbH35sbKuTlzifY3zjR5kdj8H55Mefc77VyfGzHz6GTQsDNBZQABs4ktiwKgcB9gcYQswfcKoeBaNgFIyCkQgAgwhgUwbvaNIAAAAASUVORK5CYII=","orcid":"","institution":"Moscow Engineering Physics Institute","correspondingAuthor":true,"prefix":"","firstName":"Roger","middleName":"Nick","lastName":"Anaedevha","suffix":""},{"id":390260874,"identity":"e88a50ce-364a-4b3a-a503-faefb22b8ce1","order_by":2,"name":"Alexander Genadievich Trofimov","email":"","orcid":"","institution":"Moscow Engineering Physics Institute","correspondingAuthor":false,"prefix":"","firstName":"Alexander","middleName":"Genadievich","lastName":"Trofimov","suffix":""}],"badges":[],"createdAt":"2024-11-16 00:23:06","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5463235/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5463235/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":71468234,"identity":"9cd52c93-2a60-4a2c-99ca-7c270299f75b","added_by":"auto","created_at":"2024-12-16 03:10:57","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":446455,"visible":true,"origin":"","legend":"","description":"","filename":"CMTSteganographyPapers.doc.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5463235/v1_covered_62758e08-95d8-4cc8-a628-75763b15445f.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Robust Audio-Image Steganography using Cross-Modal Based Transformer Models","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"journal-of-computer-virology-and-hacking-techniques","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jicv","sideBox":"Learn more about [Journal of Computer Virology and Hacking Techniques](http://link.springer.com/journal/11416)","snPcode":"11416","submissionUrl":"https://submission.springernature.com/new-submission/11416/3","title":"Journal of Computer Virology and Hacking Techniques","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Steganography, Deep learning, Vision Transformers, Audio transformers, Cross – Modal Transformers, Audio-to-image, Data hiding, Secure communication","lastPublishedDoi":"10.21203/rs.3.rs-5463235/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5463235/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis research investigates the use of Vision Transformers (ViT), Audio Spectrogram Transformers (AST), and Cross-Modal Transformers (CMT) in audio-image fusion tasks, aiming to improve the representation learning and interaction between auditory and visual data. The ViT model extracts visual features from image patches resized to 224x224 pixels, while the AST model converts audio signals into mel spectrograms to capture detailed auditory features. The central focus is on the robust CMT model, which integrates visual and auditory features through a cross-modal attention mechanism. Extensive experiments using a diverse audio-image dataset from Kaggle reveal significant improvements. The initial developed ViT model enhances image embedding capacity by 12%, the AST model improves audio embedding tasks by 15%, and the CMT model achieves and overall embedding capacity increased by an average of 58.7%, an image PSNR improved by an average of 57.5%, image MSE reduced by an average of 87.3% as the CMT model shows more consistent performance across all the different stegos. this enhancement in cross-modal retrieval tasks, highlighting the effectiveness of robust CMT model in learning and utilizing inter-modal relationships. The proposed robust cross-modal attention mechanism outperforms traditional ViT model and Least Significant Bits [LSB] concatenation algorithms in feature alignment accuracy. Ablation studies further validate the robustness of this approach, demonstrating the contribution of each component to overall performance. This research establishes the viability and superiority of transformer-based architectures in audio-image fusion tasks, suggesting potential advancements in cross-modal tasks like audio-visual synchronization and multimodal sentiment analysis.\u003c/p\u003e","manuscriptTitle":"Robust Audio-Image Steganography using Cross-Modal Based Transformer Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-16 03:02:49","doi":"10.21203/rs.3.rs-5463235/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"74183626160391431270849251843479175839","date":"2025-09-17T10:52:47+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-01-25T18:27:00+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"278597047324131923090306941087266666175","date":"2024-12-26T02:52:25+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-12-25T09:56:46+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-11-16T08:36:37+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-11-16T08:35:14+00:00","index":"","fulltext":""},{"type":"submitted","content":"Journal of Computer Virology and Hacking Techniques","date":"2024-11-16T00:09:15+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"journal-of-computer-virology-and-hacking-techniques","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jicv","sideBox":"Learn more about [Journal of Computer Virology and Hacking Techniques](http://link.springer.com/journal/11416)","snPcode":"11416","submissionUrl":"https://submission.springernature.com/new-submission/11416/3","title":"Journal of Computer Virology and Hacking Techniques","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"8aed2b75-0309-4b38-8b80-27d4cf58f6c2","owner":[],"postedDate":"December 16th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2024-12-16T03:02:49+00:00","versionOfRecord":[],"versionCreatedAt":"2024-12-16 03:02:49","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5463235","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5463235","identity":"rs-5463235","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}