Efficient Unsupervised Domain Adaptation via Self-Supervised Vision Transformer and Synergistic Cross-Domain Alignment

Efficient Unsupervised Domain Adaptation via Self-Supervised Vision Transformer and Synergistic Cross-Domain Alignment | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Efficient Unsupervised Domain Adaptation via Self-Supervised Vision Transformer and Synergistic Cross-Domain Alignment Ali Abedi, Q. M. Jonathan Wu, Ning Zhang, Farhad Pourpanah This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7653038/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 11 You are reading this latest preprint version Abstract Unsupervised domain adaptation (UDA) aims to mitigate the domain shift issue, where the distribution of training (source) data differs from that of testing (target) data. Many models have been developed to tackle this problem, and recently vision transformers (ViTs) have shown promising results. However, the complexity and large number of trainable parameters of ViTs restrict their deployment in practical applications. This underscores the need for an efficient model that not only reduces trainable parameters but also allows for adjustable complexity based on specific needs while delivering comparable performance. To address this challenge, we propose an Efficient Unsupervised Domain Adaptation (EUDA) framework. EUDA employs the DINOv2, which is a self-supervised ViT, as a feature extractor followed by a simplified bottleneck of fully connected layers to refine features for enhanced domain adaptation. Additionally, EUDA employs the synergistic domain alignment loss (SDAL), which integrates cross-entropy (CE) and maximum mean discrepancy (MMD) losses, to balance adaptation by minimizing classification errors in the source domain while aligning the source and target domain distributions. Experimental results demonstrate that EUDA achieves performance on par with leading UDA techniques, with significantly fewer trainable parameters, between 42% to 99.7% fewer. This showcases the ability to train the model in a resource-limited environment. Unsupervised domain adaptation Vision transformers Max mean discrepancy Self-supervised learning Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 15 Dec, 2025 Reviews received at journal 11 Dec, 2025 Reviewers agreed at journal 23 Nov, 2025 Reviews received at journal 22 Nov, 2025 Reviews received at journal 21 Oct, 2025 Reviewers agreed at journal 18 Oct, 2025 Reviewers agreed at journal 14 Oct, 2025 Reviewers invited by journal 23 Sep, 2025 Editor assigned by journal 23 Sep, 2025 Submission checks completed at journal 22 Sep, 2025 First submitted to journal 18 Sep, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7653038","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":524480985,"identity":"10869628-fe33-4b16-a0f1-cd96e68301d8","order_by":0,"name":"Ali Abedi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAzUlEQVRIiWNgGAWjYDACCSBK+GHDwMAMZhOrJbEnjVQtDGyHYWwiAP/s5oM3HvCcT9xwnPfgzR8V9xj42w8QsOTOsWSLBIvbiRsO8yVb85wpZpA4k0DAmhs5ZhIJPLcTtx3mMZNmbEtgMGAgoEX+Rv43iQS2c2Atkj9BWvgf4NdicCOHDajlAFiLBC9IiwQBWwzvHDO2SOxJNt5/mMcY6JcEHokbBGyRu9388OaPH3ayM/vPGAJDLEGOv5+ALRiAh0T1o2AUjIJRMAqwAQAud0M7FwzuyQAAAABJRU5ErkJggg==","orcid":"","institution":"University of Windsor","correspondingAuthor":true,"prefix":"","firstName":"Ali","middleName":"","lastName":"Abedi","suffix":""},{"id":524480986,"identity":"25ae3219-12d1-4146-a7e5-558402e630d6","order_by":1,"name":"Q. M. Jonathan Wu","email":"","orcid":"","institution":"University of Windsor","correspondingAuthor":false,"prefix":"","firstName":"Q.","middleName":"M. Jonathan","lastName":"Wu","suffix":""},{"id":524480987,"identity":"65f16cce-db31-482f-a48b-1a9372d575c7","order_by":2,"name":"Ning Zhang","email":"","orcid":"","institution":"University of Windsor","correspondingAuthor":false,"prefix":"","firstName":"Ning","middleName":"","lastName":"Zhang","suffix":""},{"id":524480988,"identity":"9fbdfa9d-4028-4ae0-8beb-6c9cd22c738d","order_by":3,"name":"Farhad Pourpanah","email":"","orcid":"","institution":"Queen's University","correspondingAuthor":false,"prefix":"","firstName":"Farhad","middleName":"","lastName":"Pourpanah","suffix":""}],"badges":[],"createdAt":"2025-09-18 22:23:12","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7653038/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7653038/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":92892810,"identity":"3d6c329b-1b15-459f-9968-86f5c1f61b59","added_by":"auto","created_at":"2025-10-06 18:20:23","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":3262323,"visible":true,"origin":"","legend":"","description":"","filename":"EUDA.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7653038/v1/63f8f27697e30c4b5b8875b5.pdf"},{"id":92892809,"identity":"e306f8ab-7c66-4b4d-8c6e-0707d3250730","added_by":"auto","created_at":"2025-10-06 18:20:23","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":6367,"visible":true,"origin":"","legend":"","description":"","filename":"a4235e13461041f298af276562f78440.json","url":"https://assets-eu.researchsquare.com/files/rs-7653038/v1/8e5b4d79f52432d3ec31d211.json"},{"id":92893373,"identity":"582c7406-f538-4994-8a36-79e17c757344","added_by":"auto","created_at":"2025-10-06 18:28:24","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":891473,"visible":true,"origin":"","legend":"","description":"","filename":"EUDA.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7653038/v1_covered_30529f92-f632-4f16-b543-22553b342aaf.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Efficient Unsupervised Domain Adaptation via Self-Supervised Vision Transformer and Synergistic Cross-Domain Alignment","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-machine-learning-and-cybernetics","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jmlc","sideBox":"Learn more about [International Journal of Machine Learning and Cybernetics](http://actavetscand.biomedcentral.com/)","snPcode":"13042","submissionUrl":"https://submission.nature.com/new-submission/13042/3","title":"International Journal of Machine Learning and Cybernetics","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Unsupervised domain adaptation, Vision transformers, Max mean discrepancy, Self-supervised learning","lastPublishedDoi":"10.21203/rs.3.rs-7653038/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7653038/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Unsupervised domain adaptation (UDA) aims to mitigate the domain shift issue, where the distribution of training (source) data differs from that of testing (target) data. Many models have been developed to tackle this problem, and recently vision transformers (ViTs) have shown promising results. However, the complexity and large number of trainable parameters of ViTs restrict their deployment in practical applications. This underscores the need for an efficient model that not only reduces trainable parameters but also allows for adjustable complexity based on specific needs while delivering comparable performance. To address this challenge, we propose an Efficient Unsupervised Domain Adaptation (EUDA) framework. EUDA employs the DINOv2, which is a self-supervised ViT, as a feature extractor followed by a simplified bottleneck of fully connected layers to refine features for enhanced domain adaptation. Additionally, EUDA employs the synergistic domain alignment loss (SDAL), which integrates cross-entropy (CE) and maximum mean discrepancy (MMD) losses, to balance adaptation by minimizing classification errors in the source domain while aligning the source and target domain distributions. Experimental results demonstrate that EUDA achieves performance on par with leading UDA techniques, with significantly fewer trainable parameters, between 42% to 99.7% fewer. This showcases the ability to train the model in a resource-limited environment.","manuscriptTitle":"Efficient Unsupervised Domain Adaptation via Self-Supervised Vision Transformer and Synergistic Cross-Domain Alignment","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-06 18:20:18","doi":"10.21203/rs.3.rs-7653038/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-12-15T09:07:31+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-12-12T02:53:02+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"66674683529124297889766982076858546665","date":"2025-11-23T08:19:27+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-22T13:44:24+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-10-21T16:58:05+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"270675648287864273258840156791172399482","date":"2025-10-18T13:57:05+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"108349836684400870756585683715655394626","date":"2025-10-14T12:32:14+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-09-24T01:13:11+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-09-23T10:29:58+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-09-22T05:56:02+00:00","index":"","fulltext":""},{"type":"submitted","content":"International Journal of Machine Learning and Cybernetics","date":"2025-09-18T22:12:49+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-machine-learning-and-cybernetics","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"jmlc","sideBox":"Learn more about [International Journal of Machine Learning and Cybernetics](http://actavetscand.biomedcentral.com/)","snPcode":"13042","submissionUrl":"https://submission.nature.com/new-submission/13042/3","title":"International Journal of Machine Learning and Cybernetics","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"0515055c-088a-480c-bc79-df50664aefae","owner":[],"postedDate":"October 6th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-04-16T14:11:58+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-06 18:20:18","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7653038","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7653038","identity":"rs-7653038","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}