Self-Supervised Fault Detection in Seismic Data Using a True 3D Global Attention Convolutional Network with Denoising Pretext Training

Self-Supervised Fault Detection in Seismic Data Using a True 3D Global Attention Convolutional Network with Denoising Pretext Training | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Self-Supervised Fault Detection in Seismic Data Using a True 3D Global Attention Convolutional Network with Denoising Pretext Training Matin Mahzad, Majid Bagheri This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8540420/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 12 You are reading this latest preprint version Abstract Accurate fault detection in 3D seismic data requires modeling extended geological structures across multiple dimensions. Recent CNN-transformer hybrids employ windowed attention (Swin Transformer) or factorized attention, fragmenting volumetric fault geometries into constrained local interactions that cannot fully capture continuous fault planes. Additionally, models frequently misclassify seismic noise as faults due to limited labeled data and insufficient noise exposure. We present the first application of unfactorized 3D global attention—where every voxel attends to all other voxels—integrated with volumetric convolution for seismic fault segmentation. Building on Mahzad et al. (2026), our hybrid U-Net combines 3D convolution for local features with true global self-attention for complete spatial context. Training employs two-stage learning: self-supervised denoising pretext training across multiple real-world 3D surveys, then discriminative transfer learning on fault-labeled data using layer-wise learning rate decay and Unified Focal Loss. Validation on Thebe survey data achieved Dice = 0.853, IoU = 0.744, precision = 0.842, MCC = 0.841—statistically significant improvements (p < 0.001, Cohen's d = 0.54–1.02) of 4.5–8.1% over state-of-the-art 3D CNN+Swin architecture. On unseen test data, improvements widened to 9.9–16.7% with superior generalization (3.6% vs 8.3% degradation). Results establish that unfactorized global attention with noise-aware pretext training significantly improves fault detection accuracy and generalization. Physical sciences/Engineering Physical sciences/Mathematics and computing Earth and environmental sciences/Solid earth sciences 3D seismic fault segmentation volumetric global attention hybrid convolutional-attention network self-supervised pretext training unfactorized attention Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 03 Apr, 2026 Reviews received at journal 02 Apr, 2026 Reviews received at journal 17 Mar, 2026 Reviewers agreed at journal 17 Mar, 2026 Reviewers agreed at journal 13 Mar, 2026 Reviewers agreed at journal 08 Feb, 2026 Reviewers agreed at journal 03 Feb, 2026 Reviewers invited by journal 03 Feb, 2026 Editor assigned by journal 30 Jan, 2026 Editor invited by journal 30 Jan, 2026 Submission checks completed at journal 29 Jan, 2026 First submitted to journal 28 Jan, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8540420","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":585318304,"identity":"42275476-b728-440c-bf18-59e7b6685362","order_by":0,"name":"Matin Mahzad","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA70lEQVRIiWNgGAWjYBACgwMMDAcY2xgSgGzDB0A2YS2WB5iBWs6BtRgbEKXFHqiFgfEfWIuZBFFazI6fP3i4cJtdnnz74W3VPDV35PgZmB8+uoFPy5lkhsMztyUXM/akld3mOfbMWLKBzdg4B5+WA0AtvNuYE5sZcsxu87AdTtxwgIdNGp8Wg/OPgVra6hPb+N+YFfP8I0bLDZAtbYcTeyRyzJhBDCK0PDYAajmeOEPiWbHk3L7DxpLNBPxicD7x8WfeturE+f3JGz+8+XZYjp+9+eFjfFpQABMPiGQmVjkIMP4gRfUoGAWjYBSMGAAAZONWZZu8usIAAAAASUVORK5CYII=","orcid":"","institution":"Institute of Geophysics, University of Tehran, Iran","correspondingAuthor":true,"prefix":"","firstName":"Matin","middleName":"","lastName":"Mahzad","suffix":""},{"id":585318305,"identity":"ffbe1754-61c9-4592-8f0b-30fe01ab982f","order_by":1,"name":"Majid Bagheri","email":"","orcid":"","institution":"Institute of Geophysics, University of Tehran, Iran","correspondingAuthor":false,"prefix":"","firstName":"Majid","middleName":"","lastName":"Bagheri","suffix":""}],"badges":[],"createdAt":"2026-01-07 10:53:49","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8540420/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8540420/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":101946893,"identity":"5eb2fbb8-e334-4210-b602-eefbc6724c0d","added_by":"auto","created_at":"2026-02-05 10:02:44","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2009064,"visible":true,"origin":"","legend":"","description":"","filename":"Manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8540420/v1_covered_ac10e2fb-3578-4ad0-9dbc-de28f8606e65.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Self-Supervised Fault Detection in Seismic Data Using a True 3D Global Attention Convolutional Network with Denoising Pretext Training","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"3D seismic fault segmentation, volumetric global attention, hybrid convolutional-attention network, self-supervised pretext training, unfactorized attention","lastPublishedDoi":"10.21203/rs.3.rs-8540420/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8540420/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAccurate fault detection in 3D seismic data requires modeling extended geological structures across multiple dimensions. Recent CNN-transformer hybrids employ windowed attention (Swin Transformer) or factorized attention, fragmenting volumetric fault geometries into constrained local interactions that cannot fully capture continuous fault planes. Additionally, models frequently misclassify seismic noise as faults due to limited labeled data and insufficient noise exposure.\u003c/p\u003e \u003cp\u003eWe present the first application of unfactorized 3D global attention\u0026mdash;where every voxel attends to all other voxels\u0026mdash;integrated with volumetric convolution for seismic fault segmentation. Building on Mahzad et al. (2026), our hybrid U-Net combines 3D convolution for local features with true global self-attention for complete spatial context. Training employs two-stage learning: self-supervised denoising pretext training across multiple real-world 3D surveys, then discriminative transfer learning on fault-labeled data using layer-wise learning rate decay and Unified Focal Loss.\u003c/p\u003e \u003cp\u003eValidation on Thebe survey data achieved Dice\u0026thinsp;=\u0026thinsp;0.853, IoU\u0026thinsp;=\u0026thinsp;0.744, precision\u0026thinsp;=\u0026thinsp;0.842, MCC\u0026thinsp;=\u0026thinsp;0.841\u0026mdash;statistically significant improvements (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001, Cohen's d\u0026thinsp;=\u0026thinsp;0.54\u0026ndash;1.02) of 4.5\u0026ndash;8.1% over state-of-the-art 3D CNN+Swin architecture. On unseen test data, improvements widened to 9.9\u0026ndash;16.7% with superior generalization (3.6% vs 8.3% degradation). Results establish that unfactorized global attention with noise-aware pretext training significantly improves fault detection accuracy and generalization.\u003c/p\u003e","manuscriptTitle":"Self-Supervised Fault Detection in Seismic Data Using a True 3D Global Attention Convolutional Network with Denoising Pretext Training","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-05 09:54:08","doi":"10.21203/rs.3.rs-8540420/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-04-03T12:26:50+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-03T03:34:11+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-17T06:01:03+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"51787103044246973616413854461560496277","date":"2026-03-17T05:43:46+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"165258324510445511719227465676213387055","date":"2026-03-13T18:49:02+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"193611932390356637888825072370093258737","date":"2026-02-09T03:53:17+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"70970956795501034272352409106438362735","date":"2026-02-03T23:06:12+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-02-03T16:05:18+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-01-30T15:56:20+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-01-30T12:54:38+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-01-30T03:16:29+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2026-01-28T12:59:43+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"da4933be-cd06-4a07-8547-0aae8bb896ac","owner":[],"postedDate":"February 5th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[{"id":62269318,"name":"Physical sciences/Engineering"},{"id":62269319,"name":"Physical sciences/Mathematics and computing"},{"id":62269320,"name":"Earth and environmental sciences/Solid earth sciences"}],"tags":[],"updatedAt":"2026-05-08T12:23:16+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-05 09:54:08","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8540420","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8540420","identity":"rs-8540420","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}