Channel Attention-driven Transformer with TemporalEmbedding for Oncology Care Risk Prediction

Channel Attention-driven Transformer with TemporalEmbedding for Oncology Care Risk Prediction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Channel Attention-driven Transformer with TemporalEmbedding for Oncology Care Risk Prediction Huili Du, Jing Wang Wang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7506328/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract In alignment in Computer Science’s mission to advance both foundational theories and real-world applications—particularly those that support interdisciplinary innovation and Sustainable Development Goal 9 (Industry, Innovation and Infrastructure)—this study presents a novel Channel Attention-driven Transformer architecture enhanced with temporal embeddings for predicting oncology care risk. The motivation stems from a pressing need to improve early clinical decision-making in cancer management, where traditional risk prediction models typically rely on static features or shallow temporal mechanisms. Such approaches are often insufficient for capturing the complex interdependencies among diverse physiological signals, treatment sequences, and disease trajectories in oncology patients. To overcome these challenges, our proposed framework incorporates a channel attention mechanism that adaptively reweights multimodal feature channels based on their relevance, allowing the model to selectively emphasize clinically informative signals. In parallel, we integrate temporal embeddings—both sinusoidal and learnable—to encode short-term fluctuations and long-range temporal patterns, which are crucial for modeling disease evolution. These components are embedded within multi-head self-attention Transformer blocks, further enhanced with channel-based gating to modulate signal flow at each layer. The method was rigorously evaluated on multiple real-world oncology datasets, covering diverse cancer types and risk indicators. Empirical results demonstrate that our model consistently outperforms state-of-the-art baselines across metrics including AUC, F1-score, and early warning lead time. Notably, the architecture maintains model interpretability through attention heatmaps that offer actionable insights to clinicians. This comprehensive design—blending technical novelty, clinical relevance, and transparency—illustrates how our approach contributes to impactful computational innovations in digital health, aligning seamlessly with the journal’s interdisciplinary and socially driven scope. Biological sciences/Cancer Biological sciences/Computational biology and bioinformatics Physical sciences/Mathematics and computing Channel Attention Transformer Temporal Embedding Oncology Risk Prediction Multimodal Clinical Data Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7506328","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":521692329,"identity":"c650c161-a4ee-4b4f-934c-6c4a768121ed","order_by":0,"name":"Huili Du","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABCElEQVRIiWNgGAWjYLCCBCBmbDj/8eGHCiCLHcQhRgtz4wFjY4kzDAw8zMRoAQH25gNmErxtRGgxuJFj9uBh2+HE3rYDCRKS82wS9zMzH3w4g8FOTheHPqAWc4NEoJaZPQcOGBRuS0vsYWZLNtzAkGxsdgC7FskZOWYSIC0bZxxsSJDcdhiohcdM8gHDgcRthLTsv/+Y4QDvHCK08EtAtTQ2HGNs4G2AatmATwvPszKJhHPpxo0NZ5iZJY6lGfccBvplhgFuv7CxJ2+T/FFmLQvUwv7zQ42NbHt788GHPRV2cri0MDBwGDAwsjE4NqCKGuBSDgLsDxgY/jDY41MyCkbBKBgFIxwAAApEZHVHqWFNAAAAAElFTkSuQmCC","orcid":"","institution":"Xinxiang Medical University","correspondingAuthor":true,"prefix":"","firstName":"Huili","middleName":"","lastName":"Du","suffix":""},{"id":521692330,"identity":"f2d7927e-813f-4529-8aaf-27458c4e8cfc","order_by":1,"name":"Jing Wang Wang","email":"","orcid":"","institution":"Xinxiang Medical University","correspondingAuthor":false,"prefix":"","firstName":"Jing","middleName":"Wang","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2025-09-01 08:53:46","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7506328/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7506328/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":92374608,"identity":"d1feb80a-6ae8-40c0-acc9-6fe5d4f27790","added_by":"auto","created_at":"2025-09-29 04:17:52","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5051,"visible":true,"origin":"","legend":"","description":"","filename":"dfa3a35f099b4f61b4a92aac5e141f68.json","url":"https://assets-eu.researchsquare.com/files/rs-7506328/v1/584aef54c9eb80f89e0231be.json"},{"id":93586294,"identity":"e2bf9dad-649e-42dd-ab5b-a265d1a2cfaa","added_by":"auto","created_at":"2025-10-15 11:23:53","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1880837,"visible":true,"origin":"","legend":"","description":"","filename":"SR.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7506328/v1_covered_9a0c58e9-ddd6-4ce6-a569-235f50c79502.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Channel Attention-driven Transformer with TemporalEmbedding for Oncology Care Risk Prediction","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Channel Attention, Transformer, Temporal Embedding, Oncology Risk Prediction, Multimodal Clinical Data","lastPublishedDoi":"10.21203/rs.3.rs-7506328/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7506328/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIn alignment in Computer Science’s mission to advance both foundational theories and real-world applications—particularly those that support interdisciplinary innovation and Sustainable Development Goal 9 (Industry, Innovation and Infrastructure)—this study presents a novel Channel Attention-driven Transformer architecture enhanced with temporal embeddings for predicting oncology care risk. The motivation stems from a pressing need to improve early clinical decision-making in cancer management, where traditional risk prediction models typically rely on static features or shallow temporal mechanisms. Such approaches are often insufficient for capturing the complex interdependencies among diverse physiological signals, treatment sequences, and disease trajectories in oncology patients. To overcome these challenges, our proposed framework incorporates a channel attention mechanism that adaptively reweights multimodal feature channels based on their relevance, allowing the model to selectively emphasize clinically informative signals. In parallel, we integrate temporal embeddings—both sinusoidal and learnable—to encode short-term fluctuations and long-range temporal patterns, which are crucial for modeling disease evolution. These components are embedded within multi-head self-attention Transformer blocks, further enhanced with channel-based gating to modulate signal flow at each layer. The method was rigorously evaluated on multiple real-world oncology datasets, covering diverse cancer types and risk indicators. Empirical results demonstrate that our model consistently outperforms state-of-the-art baselines across metrics including AUC, F1-score, and early warning lead time. Notably, the architecture maintains model interpretability through attention heatmaps that offer actionable insights to clinicians. This comprehensive design—blending technical novelty, clinical relevance, and transparency—illustrates how our approach contributes to impactful computational innovations in digital health, aligning seamlessly with the journal’s interdisciplinary and socially driven scope.\u003c/p\u003e","manuscriptTitle":"Channel Attention-driven Transformer with TemporalEmbedding for Oncology Care Risk Prediction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-29 04:17:47","doi":"10.21203/rs.3.rs-7506328/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"41e7bbbe-0404-46bd-8aeb-d81aae56e264","owner":[],"postedDate":"September 29th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":55450043,"name":"Biological sciences/Cancer"},{"id":55450044,"name":"Biological sciences/Computational biology and bioinformatics"},{"id":55450045,"name":"Physical sciences/Mathematics and computing"}],"tags":[],"updatedAt":"2025-10-15T11:23:19+00:00","versionOfRecord":[],"versionCreatedAt":"2025-09-29 04:17:47","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7506328","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7506328","identity":"rs-7506328","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}