COSMOS-CVNet: A Cross-modal OCR-Spam Model with Optical Stream and CNN-ViT Network for Enhanced Image Spam Detection

COSMOS-CVNet: A Cross-modal OCR-Spam Model with Optical Stream and CNN-ViT Network for Enhanced Image Spam Detection | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article COSMOS-CVNet: A Cross-modal OCR-Spam Model with Optical Stream and CNN-ViT Network for Enhanced Image Spam Detection Vaguru Mrs Swapna, A. Kethsy Prabavathy This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6401591/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Recently, spam image detection has turned out to be an increasingly intricate problem due to the evolving tactics employed by spammers, and embedding malicious content within images in sophisticated ways. Several Artificial Intelligence (AI)-aided algorithms were developed priorly intending to detect spam. However, the evolving nature of spam makes it complex for Machine Learning (ML) or Deep Learning (DL) algorithms to effectively identify them. For this reason, this paper proposes a novel hybrid model named COSMOS-CVNet, a Cross-modal OCR-Spam Model with Optical Stream and CNN-ViT Network. The proposed COSMOS-CVNet is developed by combining Convolutional Neural Network (CNN)-based image enhancement, Optical Character Recognition (OCR)-based text feature extraction, and Cross-Modal Attention (CMA) to detect spam together with enhanced CNN and Vision Transformer (ViT) networks. The model begins with preprocessing which includes CNN-based image enhancement, contrast improvement using Contrast Limited Adaptive Histogram Equalization (CLAHE), and bilateral filtering for noise reduction. The enhanced images are now passed through a pre-trained ResNet50 model to extract discriminative visual features. Simultaneously, Tesseract OCR is used to extract any embedded textual information from the images. A CMA mechanism is now used to fuse these features based on the weights of the visual and textual features to enhance model performance. Finally, the fused features are classified using a hybrid classification layer that integrates a ViT for spatial attention learning and a CNN for fine-grained feature extraction. The model is developed on a labeled dataset of spam and non-spam images, and the results of experiments demonstrate that the hybrid approach is effective in reaching superior accuracy (99%) in detecting evolving and novel spam patterns. Besides, a comparison study is carried out to evaluate the effectiveness of the proposed COSMOS-CVNet in detecting image spams over SOTA models. Image Spam Detection Deep Learning CNN ViT OCR Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6401591","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":439995846,"identity":"d062b446-19c1-4276-a98d-06f5c53b453c","order_by":0,"name":"Vaguru Mrs Swapna","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABIklEQVRIiWNgGAWjYPCDigQ4k7EBhAiDMyRrYWxD0YIdyM9Ifva4oOawvDn/GrPPvPPS5M1n5B6T+MFgI7vhAHPbAyxaDG6kmRvPOHbYcOeMN8azebflGM65kZcm2cOQZrzhAGO7ATYt0glm0jxsaYwbbpwxZubdVsE4QyLHTIKH4XAiUEubBDaHzU7/Js3zL80eomVOhT1Ii+Qfhv84tTDczjGT5m2zSdxwvgeopSEnEaRFmofhAE4tBvfflBvz9tkkb7jBVsw451ha8gyeN8bWMgbJxjMP43BYz/Ftj3m+SdhuOH94M8ObmmTbGew5hjffVNjJ9h1vf4bVYQwMbBBKIgEuwiLBAAoqZuzqEVr4D8BFmD/gVDwKRsEoGAUjEQAAjPpkDtBhNE8AAAAASUVORK5CYII=","orcid":"","institution":"Karunya University","correspondingAuthor":true,"prefix":"","firstName":"Vaguru","middleName":"Mrs","lastName":"Swapna","suffix":""},{"id":439995848,"identity":"f1f80bb3-ba12-4027-a71e-9b855695804b","order_by":1,"name":"A. Kethsy Prabavathy","email":"","orcid":"","institution":"Karunya University","correspondingAuthor":false,"prefix":"","firstName":"A.","middleName":"Kethsy","lastName":"Prabavathy","suffix":""}],"badges":[],"createdAt":"2025-04-08 09:23:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6401591/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6401591/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":80227488,"identity":"0bc29a69-f022-459c-ad91-7bcde6365518","added_by":"auto","created_at":"2025-04-09 11:54:05","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1432850,"visible":true,"origin":"","legend":"","description":"","filename":"COSMOSCVNetACrossmodalOCRSpamModelwithOpticalStreamandCNNViTNetworkforEnhancedImageSpamDetection.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6401591/v1_covered_caba6b00-12f0-4102-a29c-c8314d0ee8e5.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"COSMOS-CVNet: A Cross-modal OCR-Spam Model with Optical Stream and CNN-ViT Network for Enhanced Image Spam Detection","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Image Spam Detection, Deep Learning, CNN, ViT, OCR","lastPublishedDoi":"10.21203/rs.3.rs-6401591/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6401591/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eRecently, spam image detection has turned out to be an increasingly intricate problem due to the evolving tactics employed by spammers, and embedding malicious content within images in sophisticated ways. Several Artificial Intelligence (AI)-aided algorithms were developed priorly intending to detect spam. However, the evolving nature of spam makes it complex for Machine Learning (ML) or Deep Learning (DL) algorithms to effectively identify them. For this reason, this paper proposes a novel hybrid model named COSMOS-CVNet, a Cross-modal OCR-Spam Model with Optical Stream and CNN-ViT Network. The proposed COSMOS-CVNet is developed by combining Convolutional Neural Network (CNN)-based image enhancement, Optical Character Recognition (OCR)-based text feature extraction, and Cross-Modal Attention (CMA) to detect spam together with enhanced CNN and Vision Transformer (ViT) networks. The model begins with preprocessing which includes CNN-based image enhancement, contrast improvement using Contrast Limited Adaptive Histogram Equalization (CLAHE), and bilateral filtering for noise reduction. The enhanced images are now passed through a pre-trained ResNet50 model to extract discriminative visual features. Simultaneously, Tesseract OCR is used to extract any embedded textual information from the images. A CMA mechanism is now used to fuse these features based on the weights of the visual and textual features to enhance model performance. Finally, the fused features are classified using a hybrid classification layer that integrates a ViT for spatial attention learning and a CNN for fine-grained feature extraction. The model is developed on a labeled dataset of spam and non-spam images, and the results of experiments demonstrate that the hybrid approach is effective in reaching superior accuracy (99%) in detecting evolving and novel spam patterns. Besides, a comparison study is carried out to evaluate the effectiveness of the proposed COSMOS-CVNet in detecting image spams over SOTA models.\u003c/p\u003e","manuscriptTitle":"COSMOS-CVNet: A Cross-modal OCR-Spam Model with Optical Stream and CNN-ViT Network for Enhanced Image Spam Detection","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-09 05:28:57","doi":"10.21203/rs.3.rs-6401591/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"48efe6a1-c09f-4550-aeca-531ec2006cd4","owner":[],"postedDate":"April 9th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-04-09T11:53:21+00:00","versionOfRecord":[],"versionCreatedAt":"2025-04-09 05:28:57","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6401591","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6401591","identity":"rs-6401591","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}