Comparative analysis of Attention-based CNN andViT for processing ECG image data in joint fusion

Comparative analysis of Attention-based CNN andViT for processing ECG image data in joint fusion | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Comparative analysis of Attention-based CNN andViT for processing ECG image data in joint fusion Arjun Thakur, Pradyumna Agasthi, Chieh-Ju Chao, Juan Maria Farina, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3892330/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Percutaneous Coronary Intervention (PCI) is a widely utilized standard ofcare and efficacious intervention in the management of coronary artery disease,improving prognosis and symptoms for many patients. Prediction of post-PCIoutcomes contributes to effective patient management and quality improvementinitiatives in healthcare. Our study focuses on comparative performance analysisfor state-of-the-art vision transformer (ViT) and convolutional neural network(CNN) on multimodal data analysis in a joint fusion framework. We integrateimages of electrocardiogram (ECG) data and tabular electronic health records(EHR) to predict 3 clinically relevant post-PCI (6 months) endpoints - heart failure hospitalization, all-cause mortality, and ischemic stroke. To design a comparativemodel for ViT, we proposed a new joint fusion architecture that consistsof a convolutional neural network (CNN) with a convolutional block attentionmodule (CBAM). The learned representations are combined at an intermediatelayer followed by processing these representations using a fully connectedlayer. The proposed model demonstrates excellent performance with the highestAUROC scores of 0.849, 0.913, and 0.794 for predicting heart failure hospitalization,all-cause mortality, and stroke, respectively. Surpassing the baseline EHRmodel and ViT, the proposed CNN+CBAM fusion model showcases superiorpredictive capabilities for heart failure prediction. Adverse Cardiovascular Outcome Convolutional Block Attention Module Joint Fusion Percutaneous Coronary Intervention Vision Transformer Full Text Additional Declarations No competing interests reported. Supplementary Files Supplement.pdf Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3892330","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":269190137,"identity":"7ec23eac-3274-4afa-ad4a-ed842f5ad81f","order_by":0,"name":"Arjun Thakur","email":"","orcid":"","institution":"Vishwakarma University","correspondingAuthor":false,"prefix":"","firstName":"Arjun","middleName":"","lastName":"Thakur","suffix":""},{"id":269190138,"identity":"9b798e5f-2ee6-48af-94c9-334207af471c","order_by":1,"name":"Pradyumna Agasthi","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Pradyumna","middleName":"","lastName":"Agasthi","suffix":""},{"id":269190139,"identity":"75ebff6c-6ff1-4e9c-aa28-6a01f7398384","order_by":2,"name":"Chieh-Ju Chao","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Chieh-Ju","middleName":"","lastName":"Chao","suffix":""},{"id":269190140,"identity":"4a29b629-f501-4bbb-ab1c-89e8b1736c1a","order_by":3,"name":"Juan Maria Farina","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Juan","middleName":"Maria","lastName":"Farina","suffix":""},{"id":269190141,"identity":"477adbf7-e91c-4788-9327-7f23f4f47bc7","order_by":4,"name":"David R. Holmes","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"David","middleName":"R.","lastName":"Holmes","suffix":""},{"id":269190142,"identity":"2aa6831a-decc-4d07-9603-d2d4571adea8","order_by":5,"name":"David Fortuin","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"David","middleName":"","lastName":"Fortuin","suffix":""},{"id":269190143,"identity":"221f3ae4-6716-4fd8-b884-bb33745794b1","order_by":6,"name":"Chadi Ayoub","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Chadi","middleName":"","lastName":"Ayoub","suffix":""},{"id":269190144,"identity":"b1ea245b-620c-4fc5-b965-39f447879dbd","order_by":7,"name":"Reza Arsanjani","email":"","orcid":"","institution":"Mayo Clinic","correspondingAuthor":false,"prefix":"","firstName":"Reza","middleName":"","lastName":"Arsanjani","suffix":""},{"id":269190145,"identity":"e210b458-d030-4037-984f-229931cfd45e","order_by":8,"name":"Imon Banerjee","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABAklEQVRIiWNgGAWjYBACezDJA8T8B4BEBVSYB48WwwaYCmYQ4wwRWgwOwFggLYxtxNjSfvbg4woZGzkGZu7Ex4XztskbHG9gfPC2DbcWe568ZMMzPGnGDMy8m41nbrttuOHMAWbDuXi0GDbkmEk28BxObGDm3SbNu+0244YbCWzSvHi0GJx/Y/6zged/PUTLnNv2QC3sv/FquZFjxtjAcyCBAayl4XYiyBZmfFoMZ7wxBjos2bAN5BeeY7eTZ5452Cw55xwe7/PnGH5s7LGT5+c/u/ExT81t277jzQc/vCnDrQUMGHsYGNhgHIUDjA0E1IPADyS2PDEaRsEoGAWjYEQBAEG1US7kqfciAAAAAElFTkSuQmCC","orcid":"","institution":"Mayo Clinic","correspondingAuthor":true,"prefix":"","firstName":"Imon","middleName":"","lastName":"Banerjee","suffix":""}],"badges":[],"createdAt":"2024-01-24 00:00:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3892330/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3892330/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":60779826,"identity":"9ee5a773-0dd8-4682-9d21-7c75fb4c3b92","added_by":"auto","created_at":"2024-07-22 01:47:41","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2609785,"visible":true,"origin":"","legend":"","description":"","filename":"PredictionofAdverseCardiovascularoutcomespringernature.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3892330/v1_covered_bfa2e31b-d733-402d-ba21-5c8b19d1a7f6.pdf"},{"id":50295926,"identity":"3c202eeb-e749-42f4-b6da-da5a30de9983","added_by":"auto","created_at":"2024-01-29 10:03:08","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":5948686,"visible":true,"origin":"","legend":"","description":"","filename":"Supplement.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3892330/v1/96355f158dfc934c82b167e0.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Comparative analysis of Attention-based CNN andViT for processing ECG image data in joint fusion","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Adverse Cardiovascular Outcome, Convolutional Block Attention Module, Joint Fusion, Percutaneous Coronary Intervention, Vision Transformer","lastPublishedDoi":"10.21203/rs.3.rs-3892330/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3892330/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Percutaneous Coronary Intervention (PCI) is a widely utilized standard ofcare and efficacious intervention in the management of coronary artery disease,improving prognosis and symptoms for many patients. Prediction of post-PCIoutcomes contributes to effective patient management and quality improvementinitiatives in healthcare. Our study focuses on comparative performance analysisfor state-of-the-art vision transformer (ViT) and convolutional neural network(CNN) on multimodal data analysis in a joint fusion framework. We integrateimages of electrocardiogram (ECG) data and tabular electronic health records(EHR) to predict 3 clinically relevant post-PCI (6 months) endpoints - heart failure hospitalization, all-cause mortality, and ischemic stroke. To design a comparativemodel for ViT, we proposed a new joint fusion architecture that consistsof a convolutional neural network (CNN) with a convolutional block attentionmodule (CBAM). The learned representations are combined at an intermediatelayer followed by processing these representations using a fully connectedlayer. The proposed model demonstrates excellent performance with the highestAUROC scores of 0.849, 0.913, and 0.794 for predicting heart failure hospitalization,all-cause mortality, and stroke, respectively. Surpassing the baseline EHRmodel and ViT, the proposed CNN+CBAM fusion model showcases superiorpredictive capabilities for heart failure prediction.","manuscriptTitle":"Comparative analysis of Attention-based CNN andViT for processing ECG image data in joint fusion","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-01-29 10:03:03","doi":"10.21203/rs.3.rs-3892330/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"402dafeb-7f5e-4c5e-b014-6d2523d39fec","owner":[],"postedDate":"January 29th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-07-22T01:39:28+00:00","versionOfRecord":[],"versionCreatedAt":"2024-01-29 10:03:03","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3892330","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3892330","identity":"rs-3892330","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}