Lightweight Transformer Models for Biomedical Signal Processing: Trends, Challenges, and Future Directions

Lightweight Transformer Models for Biomedical Signal Processing: Trends, Challenges, and Future Directions | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF survey Lightweight Transformer Models for Biomedical Signal Processing: Trends, Challenges, and Future Directions Vallem Gopi This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7620509/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Transformers, initially transformative in natural language processing and later in vision, are now rapidly gaining traction in biomedical signal processing. Electrocardiograms (ECG), electroencephalograms (EEG), and electromyograms (EMG) are critical for diagnosing cardiovascular, neurological, and neuromuscular conditions. Yet, canonical Transformers remain computationally heavy, hindering deployment in wearables and embedded devices that require real-time, low-power inference. This paper presents the first structured survey (2023–2025) of lightweight Transformer models for biosignals, introducing a novel taxonomy of efficiency strategies, including architectural compaction, efficient attention, pruning, quantization, distillation, hybrids, and hardware-aware neural architecture search. We review nearly 100 recent works across ECG, EEG, and EMG, providing comparative analysis of model size, FLOPs/MACs, latency, and accuracy. A case study on the MIT-BIH dataset demonstrates that a compact Transformer achieves 98.40% accuracy with sub-millisecond latency, outperforming a CNN baseline and reducing false negatives, which are critical in clinical settings. To ensure reproducibility, the full implementation and training scripts are made available in an open-access Google Colab notebook. We conclude with open challenges—dataset scale and bias, subject-wise generalization, interpretability, and deployment trade-offs—and propose a roadmap for multimodal, federated, and explainable biosignal Transformers optimized for next-generation digital health. Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7620509","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"survey","associatedPublications":[],"authors":[{"id":523815227,"identity":"311030ea-885f-428d-8830-a33edf143493","order_by":0,"name":"Vallem Gopi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6UlEQVRIie3RPQrCMBiA4YRIXKJdLf6cQShUXPQspWs3QRC1RpzF2Vs4fXNDwC4eQHCxeoIOgg6CiTo3HQXzLl+GPCQhCNlsPxlZJp9JxFkNVjcTzL+Ehl1NaHmCmN/Qw0i6qeBJHs07vSrzJ7do0KKIZJdjETkEXGwh9forOj61IVQXo54XFRCXB1zWYB/sJIGTC0QRRpuFZJNx+XwT5I9cWJiJ01CnYJgpUvFxDrIMybhYQ6LfEjYxpIwSw1uoE4rzA+JOz5Eif8B06FRX2bWIfJNIfyNhek3M23Xxm+B7ud02m832Z70AfNBLpy3AMlsAAAAASUVORK5CYII=","orcid":"","institution":"Prasad V. Potluri Siddhartha Institute of Technology Vijayawada","correspondingAuthor":true,"prefix":"","firstName":"Vallem","middleName":"","lastName":"Gopi","suffix":""}],"badges":[],"createdAt":"2025-09-15 12:23:25","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7620509/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7620509/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":92972615,"identity":"7a9ba2df-a4d1-4157-9a4f-c4c56ef6126f","added_by":"auto","created_at":"2025-10-07 17:12:28","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":4091,"visible":true,"origin":"","legend":"","description":"","filename":"ad7e9520dc394030994baf768408c629.json","url":"https://assets-eu.researchsquare.com/files/rs-7620509/v1/8bfd782088cc959b0d4f120a.json"},{"id":95312564,"identity":"bed6b69d-7a2a-4a2f-91f5-e5be2326dbd7","added_by":"auto","created_at":"2025-11-06 15:49:45","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":482766,"visible":true,"origin":"","legend":"","description":"","filename":"LightweightTransformerModelsforBiomedicalSignalProcessingTrendsChallengesandFutureDirections12.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7620509/v1_covered_7fbf84b6-4abd-4107-b59b-e2ccc8ed8e87.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Lightweight Transformer Models for Biomedical Signal Processing: Trends, Challenges, and Future Directions","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7620509/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7620509/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTransformers, initially transformative in natural language processing and later in vision, are now rapidly gaining traction in biomedical signal processing. Electrocardiograms (ECG), electroencephalograms (EEG), and electromyograms (EMG) are critical for diagnosing cardiovascular, neurological, and neuromuscular conditions. Yet, canonical Transformers remain computationally heavy, hindering deployment in wearables and embedded devices that require real-time, low-power inference.\u003c/p\u003e\n\u003cp\u003eThis paper presents the first structured survey (2023–2025) of lightweight Transformer models for biosignals, introducing a novel taxonomy of efficiency strategies, including architectural compaction, efficient attention, pruning, quantization, distillation, hybrids, and hardware-aware neural architecture search. We review nearly 100 recent works across ECG, EEG, and EMG, providing comparative analysis of model size, FLOPs/MACs, latency, and accuracy.\u003c/p\u003e\n\u003cp\u003eA case study on the MIT-BIH dataset demonstrates that a compact Transformer achieves 98.40% accuracy with sub-millisecond latency, outperforming a CNN baseline and reducing false negatives, which are critical in clinical settings. To ensure reproducibility, the full implementation and training scripts are made available in an open-access Google Colab notebook.\u003c/p\u003e\n\u003cp\u003eWe conclude with open challenges—dataset scale and bias, subject-wise generalization, interpretability, and deployment trade-offs—and propose a roadmap for multimodal, federated, and explainable biosignal Transformers optimized for next-generation digital health.\u003c/p\u003e","manuscriptTitle":"Lightweight Transformer Models for Biomedical Signal Processing: Trends, Challenges, and Future Directions","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-07 17:12:24","doi":"10.21203/rs.3.rs-7620509/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"458a83c7-080b-4e0e-b53a-ff7a148bf769","owner":[],"postedDate":"October 7th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-11-05T17:08:44+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-07 17:12:24","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7620509","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7620509","identity":"rs-7620509","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}