SPSTNet: Image Super-Resolution Using Spatial Pyramid Swin Transformer Network

SPSTNet: Image Super-Resolution Using Spatial Pyramid Swin Transformer Network | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article SPSTNet: Image Super-Resolution Using Spatial Pyramid Swin Transformer Network Yemei Sun, Jiao Wang, Yue Yang, Yan Zhang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5432896/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 12 Feb, 2025 Read the published version in Signal, Image and Video Processing → Version 1 posted 9 You are reading this latest preprint version Abstract Recent research on enhancing image resolution using convolutional neural networks (CNNs) have shown encouraging outcomes. While due to the intrinsic locality of the convolution operator, CNN-based methods limit the capacity to obtain contextual information and long-range dependency. To address this problem, we propose a hybrid network by integrating CNN and Transformer which show impressive performance to learn long-range contextual information for image SR. Specifically, by introducing a spatial pyramid pooling (SPP) module into the multi-head attention (MHA), the Spatial Pyramid Swin Transformer (SPST) module achieves linear computational complexity and integrates multi-scale features. This enables the model to learn a wider range of multi-scale features and enhances the capabilities of the attention matrix. Moreover, the gated convolution (GC) module employs the abundant low-frequency information from low-resolution to assist reconstruction and provides a learnable dynamic feature selection mechanism to further constrain the training to improve performance. Extensive experiments were carried out to assess the efficacy of our approach utilizing a benchmark dataset. The results of indicate that our method surpasses alternative approaches in terms of parameter count and computational efficiency. Especially, the proposed method increases PSNR by 0.05 dB and uses 1.6M fewer parameters than SwinIR, resulting in a shorter inference time. Super-resolution Residual connection Convolutional neural network Transformer Spatial pyramid attention Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 12 Feb, 2025 Read the published version in Signal, Image and Video Processing → Version 1 posted Editorial decision: Revision requested 09 Dec, 2024 Reviews received at journal 25 Nov, 2024 Reviewers agreed at journal 25 Nov, 2024 Reviews received at journal 20 Nov, 2024 Reviewers agreed at journal 20 Nov, 2024 Reviewers invited by journal 20 Nov, 2024 Editor assigned by journal 12 Nov, 2024 Submission checks completed at journal 12 Nov, 2024 First submitted to journal 11 Nov, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5432896","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":382559413,"identity":"f3ea3de7-584a-4867-a657-7585cadaa5d5","order_by":0,"name":"Yemei Sun","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7ElEQVRIiWNgGAWjYBADHjb2xsYHHwxs7IjXwsdzuNlwRkFaMvHWyEmktwnzfDjE2EBIpfyM3GMSH3NsZNgYEtuYbQwOMDOwHz66AZ8Wgxt5aZIzt6XxsDEcbHucY3CHj4EnLe0GXi0SOWbSvNsO87AxNrYb5xg8Y2aQ4DHDq0V+BlDL323/ediYGdukLQwOMzYQ0sJwA6iFcdsBHjY2oBYGYrQYnHljbNm7LZmHjYex2bDHIC2ZjZBf5NtzDG/83GZnLz//+cMHP/7Y2PGzHz6G32EMDCwSKFw2AspBgPkDEYpGwSgYBaNgJAMAV0hGEOfTL6oAAAAASUVORK5CYII=","orcid":"","institution":"Tianjin Chengjian University","correspondingAuthor":true,"prefix":"","firstName":"Yemei","middleName":"","lastName":"Sun","suffix":""},{"id":382559414,"identity":"1c59a656-8c9f-4b6b-acd1-c8db17a0842e","order_by":1,"name":"Jiao Wang","email":"","orcid":"","institution":"Tianjin Chengjian University","correspondingAuthor":false,"prefix":"","firstName":"Jiao","middleName":"","lastName":"Wang","suffix":""},{"id":382559415,"identity":"b0ccb50b-f2ef-472b-973f-740c05cc45f8","order_by":2,"name":"Yue Yang","email":"","orcid":"","institution":"Tianjin Chengjian University","correspondingAuthor":false,"prefix":"","firstName":"Yue","middleName":"","lastName":"Yang","suffix":""},{"id":382559416,"identity":"add28242-be7b-418d-ad8d-d63f773f31e4","order_by":3,"name":"Yan Zhang","email":"","orcid":"","institution":"Tianjin Chengjian University","correspondingAuthor":false,"prefix":"","firstName":"Yan","middleName":"","lastName":"Zhang","suffix":""}],"badges":[],"createdAt":"2024-11-11 14:38:27","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5432896/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5432896/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s11760-025-03880-8","type":"published","date":"2025-02-12T15:57:35+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":76487566,"identity":"a0669ddf-ef5a-4282-b010-02c39e26ee1d","added_by":"auto","created_at":"2025-02-17 16:09:15","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":11148626,"visible":true,"origin":"","legend":"","description":"","filename":"snarticle.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5432896/v1_covered_90265194-7ec9-4341-8363-b421cfd9b53c.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"SPSTNet: Image Super-Resolution Using Spatial Pyramid Swin Transformer Network","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"signal-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"sivp","sideBox":"Learn more about [Signal, Image and Video Processing](http://link.springer.com/journal/11760)","snPcode":"11760","submissionUrl":"https://submission.nature.com/new-submission/11760/3","title":"Signal, Image and Video Processing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Super-resolution, Residual connection, Convolutional neural network, Transformer, Spatial pyramid attention","lastPublishedDoi":"10.21203/rs.3.rs-5432896/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5432896/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Recent research on enhancing image resolution using convolutional neural networks (CNNs) have shown encouraging outcomes. While due to the intrinsic locality of the convolution operator, CNN-based methods limit the capacity to obtain contextual information and long-range dependency. To address this problem, we propose a hybrid network by integrating CNN and Transformer which show impressive performance to learn long-range contextual information for image SR. Specifically, by introducing a spatial pyramid pooling (SPP) module into the multi-head attention (MHA), the Spatial Pyramid Swin Transformer (SPST) module achieves linear computational complexity and integrates multi-scale features. This enables the model to learn a wider range of multi-scale features and enhances the capabilities of the attention matrix. Moreover, the gated convolution (GC) module employs the abundant low-frequency information from low-resolution to assist reconstruction and provides a learnable dynamic feature selection mechanism to further constrain the training to improve performance. Extensive experiments were carried out to assess the efficacy of our approach utilizing a benchmark dataset. The results of indicate that our method surpasses alternative approaches in terms of parameter count and computational efficiency. Especially, the proposed method increases PSNR by 0.05 dB and uses 1.6M fewer parameters than SwinIR, resulting in a shorter inference time.","manuscriptTitle":"SPSTNet: Image Super-Resolution Using Spatial Pyramid Swin Transformer Network","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-11-29 16:15:59","doi":"10.21203/rs.3.rs-5432896/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-12-10T01:39:39+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-25T22:09:09+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"149316033708863779802410340962590776592","date":"2024-11-25T21:30:07+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-20T06:24:13+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"203366863475633198008779760711758038373","date":"2024-11-20T06:07:57+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-11-20T05:04:01+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-11-12T08:24:18+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-11-12T08:23:38+00:00","index":"","fulltext":""},{"type":"submitted","content":"Signal, Image and Video Processing","date":"2024-11-11T14:33:58+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"signal-image-and-video-processing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"sivp","sideBox":"Learn more about [Signal, Image and Video Processing](http://link.springer.com/journal/11760)","snPcode":"11760","submissionUrl":"https://submission.nature.com/new-submission/11760/3","title":"Signal, Image and Video Processing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"97977632-90f3-42eb-9dd7-f416a73f0244","owner":[],"postedDate":"November 29th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-02-17T16:02:34+00:00","versionOfRecord":{"articleIdentity":"rs-5432896","link":"https://doi.org/10.1007/s11760-025-03880-8","journal":{"identity":"signal-image-and-video-processing","isVorOnly":false,"title":"Signal, Image and Video Processing"},"publishedOn":"2025-02-12 15:57:35","publishedOnDateReadable":"February 12th, 2025"},"versionCreatedAt":"2024-11-29 16:15:59","video":"","vorDoi":"10.1007/s11760-025-03880-8","vorDoiUrl":"https://doi.org/10.1007/s11760-025-03880-8","workflowStages":[]},"version":"v1","identity":"rs-5432896","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5432896","identity":"rs-5432896","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}