MSA-MVSNet: A Cross-Scale Collaborative Attention-Based Multi-View Reconstruction Network for Orchard Tree 3D Reconstruction with Instance Segmentation for Fruit Counting

MSA-MVSNet: A Cross-Scale Collaborative Attention-Based Multi-View Reconstruction Network for Orchard Tree 3D Reconstruction with Instance Segmentation for Fruit Counting | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article MSA-MVSNet: A Cross-Scale Collaborative Attention-Based Multi-View Reconstruction Network for Orchard Tree 3D Reconstruction with Instance Segmentation for Fruit Counting Hui Li, Jun Zhang, Jianhua Hong, Fuzhi Ke, Chu Zhang, Zhixin Li This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8759462/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract To address the issues of detail loss and matching difficulties in fruit tree 3D reconstruction caused by complex branch–leaf morphology, fruit occlusion, and illumination variations, this paper proposes an end-to-end cross-scale collaborative attention multi-view stereo network, termed MSA-MVSNet, for high-quality 3D reconstruction of orchard trees, while integrating semantic segmentation for fruit counting. A multi-scale feature enhancement module is designed to adaptively fuse deep semantic features and shallow fine-grained details through a spatial–channel collaborative attention mechanism, thereby enhancing the network’s capability to represent multi-scale structures such as trunks, branches, and leaves. Multi-branch dilated convolutions are introduced to enlarge the receptive field, and deformable convolutions are incorporated to adaptively capture the irregular geometric shapes of fruits, improving modeling robustness. In addition, a feature matching transformer is introduced to strengthen long-range global contextual correlations within and across images via intra-attention and inter-attention mechanisms, thereby improving matching stability in low-texture and repetitive-texture regions.To validate the effectiveness of the proposed method, experiments are conducted on self-collected real orchard dataset and public benchmark datasets. The results demonstrate that MSA-MVSNet outperforms baseline models by 8.2% in terms of 3D reconstruction quality. Finally, by combining depth filtering with the semantic segmentation results of YOLOv11-Seg, a semantic-guided fruit reconstruction and counting framework is constructed. This framework achieves an overall counting F1-score of 92.8% on the self-collected dataset with varying scene sparsity and 93.5% on the public Fuji-sfm dataset, demonstrating its effectiveness and generalization capability. Multi-View Stereo 3D Reconstruction Semantic Segmentation Fruit Counting Attention Mechanism Smart Agriculture Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviewers agreed at journal 17 Mar, 2026 Reviewers invited by journal 02 Mar, 2026 Editor assigned by journal 23 Feb, 2026 Submission checks completed at journal 02 Feb, 2026 First submitted to journal 01 Feb, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8759462","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":599690439,"identity":"0690fb43-a7eb-456b-924e-108414bc40bb","order_by":0,"name":"Hui Li","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1UlEQVRIiWNgGAWjYBACeWbmw4//VNTI2R9vPkCcFsP2tjQDnjPHjBnOHEsg0pozZwwkeFuYExtu+BgQp4NxRoKBgWQDmzHjDJ6PN94w2MnpNhDQwi6RkPDAcIeMHLN072bLOQzJxmYHCNtywCDxDJsxm8zZbdI8DAcStxHSwnAjsUHiYBtzYo9EzjMitZw5zCDZCNQyQyKHjTgtwEBmAwWwsQHPMWPLOQZE+EWemf/zYwZgVBqwNz+88abCTo6gFhQgwUNk1CBrIVXHKBgFo2AUjAgAABWHRCNoZRjIAAAAAElFTkSuQmCC","orcid":"","institution":"Huzhou University","correspondingAuthor":true,"prefix":"","firstName":"Hui","middleName":"","lastName":"Li","suffix":""},{"id":599690447,"identity":"4593b1cf-ee37-40cf-93a3-3ecb17aa0482","order_by":1,"name":"Jun Zhang","email":"","orcid":"","institution":"ZheJiang Academy of Agricultural Sciences","correspondingAuthor":false,"prefix":"","firstName":"Jun","middleName":"","lastName":"Zhang","suffix":""},{"id":599690448,"identity":"f9bbb097-c0ee-4280-ad09-aa53d4d4a251","order_by":2,"name":"Jianhua Hong","email":"","orcid":"","institution":"Huzhou University","correspondingAuthor":false,"prefix":"","firstName":"Jianhua","middleName":"","lastName":"Hong","suffix":""},{"id":599690449,"identity":"eeb9d4e9-3d4e-4ec1-9b9f-f773067d12a0","order_by":3,"name":"Fuzhi Ke","email":"","orcid":"","institution":"Zhejiang Citrus Research Institute","correspondingAuthor":false,"prefix":"","firstName":"Fuzhi","middleName":"","lastName":"Ke","suffix":""},{"id":599690453,"identity":"156b60bf-343c-4d24-9607-ffa7cb779736","order_by":4,"name":"Chu Zhang","email":"","orcid":"","institution":"Huzhou University","correspondingAuthor":false,"prefix":"","firstName":"Chu","middleName":"","lastName":"Zhang","suffix":""},{"id":599690455,"identity":"5890f8ba-9c55-4ad8-8670-1810b3ae5034","order_by":5,"name":"Zhixin Li","email":"","orcid":"","institution":"ZheJiang Academy of Agricultural Sciences","correspondingAuthor":false,"prefix":"","firstName":"Zhixin","middleName":"","lastName":"Li","suffix":""}],"badges":[],"createdAt":"2026-02-02 01:53:13","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8759462/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8759462/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":103976937,"identity":"a4809b06-4eff-4375-ac20-1ff8a4ded33d","added_by":"auto","created_at":"2026-03-05 08:48:43","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1865802,"visible":true,"origin":"","legend":"","description":"","filename":"revisedmanuscript20260203.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8759462/v1_covered_e262c31c-77cb-4f17-ba66-a3d2e81d69bb.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"MSA-MVSNet: A Cross-Scale Collaborative Attention-Based Multi-View Reconstruction Network for Orchard Tree 3D Reconstruction with Instance Segmentation for Fruit Counting","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"multimedia-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"mmsj","sideBox":"Learn more about [Multimedia Systems](http://link.springer.com/journal/530)","snPcode":"530","submissionUrl":"https://submission.nature.com/new-submission/530/3","title":"Multimedia Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Multi-View Stereo, 3D Reconstruction, Semantic Segmentation, Fruit Counting, Attention Mechanism, Smart Agriculture","lastPublishedDoi":"10.21203/rs.3.rs-8759462/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8759462/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTo address the issues of detail loss and matching difficulties in fruit tree 3D reconstruction caused by complex branch\u0026ndash;leaf morphology, fruit occlusion, and illumination variations, this paper proposes an end-to-end cross-scale collaborative attention multi-view stereo network, termed MSA-MVSNet, for high-quality 3D reconstruction of orchard trees, while integrating semantic segmentation for fruit counting. A multi-scale feature enhancement module is designed to adaptively fuse deep semantic features and shallow fine-grained details through a spatial\u0026ndash;channel collaborative attention mechanism, thereby enhancing the network\u0026rsquo;s capability to represent multi-scale structures such as trunks, branches, and leaves. Multi-branch dilated convolutions are introduced to enlarge the receptive field, and deformable convolutions are incorporated to adaptively capture the irregular geometric shapes of fruits, improving modeling robustness. In addition, a feature matching transformer is introduced to strengthen long-range global contextual correlations within and across images via intra-attention and inter-attention mechanisms, thereby improving matching stability in low-texture and repetitive-texture regions.To validate the effectiveness of the proposed method, experiments are conducted on self-collected real orchard dataset and public benchmark datasets. The results demonstrate that MSA-MVSNet outperforms baseline models by 8.2% in terms of 3D reconstruction quality. Finally, by combining depth filtering with the semantic segmentation results of YOLOv11-Seg, a semantic-guided fruit reconstruction and counting framework is constructed. This framework achieves an overall counting F1-score of 92.8% on the self-collected dataset with varying scene sparsity and 93.5% on the public Fuji-sfm dataset, demonstrating its effectiveness and generalization capability.\u003c/p\u003e","manuscriptTitle":"MSA-MVSNet: A Cross-Scale Collaborative Attention-Based Multi-View Reconstruction Network for Orchard Tree 3D Reconstruction with Instance Segmentation for Fruit Counting","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-05 08:48:31","doi":"10.21203/rs.3.rs-8759462/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"191650914853201248566712922965953123017","date":"2026-03-17T10:14:31+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-03-02T09:48:43+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-02-23T10:09:42+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-02-03T04:13:03+00:00","index":"","fulltext":""},{"type":"submitted","content":"Multimedia Systems","date":"2026-02-02T01:42:58+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"multimedia-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"mmsj","sideBox":"Learn more about [Multimedia Systems](http://link.springer.com/journal/530)","snPcode":"530","submissionUrl":"https://submission.nature.com/new-submission/530/3","title":"Multimedia Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"4f089e6b-ebd2-42fc-951a-23aede89ed31","owner":[],"postedDate":"March 5th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-03-05T08:48:32+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-05 08:48:31","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8759462","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8759462","identity":"rs-8759462","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}