A unifying view of contrastive learning, importance sampling, and bridge sampling for energy-based models

A unifying view of contrastive learning, importance sampling, and bridge sampling for energy-based models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A unifying view of contrastive learning, importance sampling, and bridge sampling for energy-based models Luca Martino This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9368004/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 6 You are reading this latest preprint version Abstract In the last decades, energy-based models (EBMs) have become an important class of probabilistic models in which a component of the likelihood is intractable and therefore cannot be evaluated explicitly. Consequently, parameter estimation in EBMs is challenging for conventional inference methods. In this work, we provide a unified framework that connects noise contrastive estimation (NCE), reverse logistic regression (RLR), multiple importance sampling (MIS), and bridge sampling within the context of EBMs. We further show that these methods are equivalent under specific conditions. This unified perspective clarifies relationships among existing methods and enables the development of new estimators, with the potential to improve statistical and computational efficiency. Furthermore, this study helps elucidate the success of NCE in terms of its flexibility and robustness, while also identifying scenarios in which its performance can be further improved. Hence, rather than being a purely descriptive review, this work offers a unifying perspective and additional methodological contributions. The MATLAB code used in the numerical experiments is also made freely available to support the reproducibility of the results. Contrastive learning bridge sampling reverse logistic regression multiple importance sampling binary classification Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviewers agreed at journal 16 Apr, 2026 Reviewers agreed at journal 15 Apr, 2026 Reviewers invited by journal 14 Apr, 2026 Editor assigned by journal 11 Apr, 2026 Submission checks completed at journal 11 Apr, 2026 First submitted to journal 09 Apr, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9368004","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":624151397,"identity":"0f4d24b2-2828-488d-9ef8-04448ac30e86","order_by":0,"name":"Luca Martino","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA10lEQVRIiWNgGAWjYBACPjiLGYg/MDDwgNgH8GlhgzF4gFoYZ4C0AIUO4NOD0AKyiAcmhFeLRPLDD4w5dvn27LwHH9u23ZHhn9/AePgDXi1pxhKM25Ite5j5ko1z257xSBwj5DCJHAagFmYDHmYeM+nctsM8BgT9IpHD/INxWz1EiyWRWtiAthyGaGEkSgvPMzOLxG3HDXgO8xgb9pw7DPRLYsOBM3i08LMnP77xcVu1AXv/GcMHP8oO2/M3Hz78oQKPFjBIQOUyNhDSMApGwSgYBaOAAAAAFRxAU9eLqJwAAAAASUVORK5CYII=","orcid":"","institution":"University of Catania","correspondingAuthor":true,"prefix":"","firstName":"Luca","middleName":"","lastName":"Martino","suffix":""}],"badges":[],"createdAt":"2026-04-09 11:38:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9368004/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9368004/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":107424172,"identity":"17ab8699-fe1b-4417-bb4e-f5242a4ef504","added_by":"auto","created_at":"2026-04-21 10:57:13","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":425344,"visible":true,"origin":"","legend":"","description":"","filename":"BRIDGEsamplingNCEv5.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9368004/v1_covered_e4cbec80-a120-41cc-a262-eb8f6a7158ce.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A unifying view of contrastive learning, importance sampling, and bridge sampling for energy-based models","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"statistics-and-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"stco","sideBox":"Learn more about [Statistics and Computing](http://link.springer.com/journal/11222)","snPcode":"11222","submissionUrl":"https://submission.nature.com/new-submission/11222/3","title":"Statistics and Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Contrastive learning, bridge sampling; reverse logistic regression, multiple importance sampling, binary classification","lastPublishedDoi":"10.21203/rs.3.rs-9368004/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9368004/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"In the last decades, energy-based models (EBMs) have become an important class of probabilistic models in which a component of the likelihood is intractable and therefore cannot be evaluated explicitly. Consequently, parameter estimation in EBMs is challenging for conventional inference methods. In this work, we provide a unified framework that connects noise contrastive estimation (NCE), reverse logistic regression (RLR), multiple importance sampling (MIS), and bridge sampling within the context of EBMs. We further show that these methods are equivalent under specific conditions. This unified perspective clarifies relationships among existing methods and enables the development of new estimators, with the potential to improve statistical and computational efficiency. Furthermore, this study helps elucidate the success of NCE in terms of its flexibility and robustness, while also identifying scenarios in which its performance can be further improved. Hence, rather than being a purely descriptive review, this work offers a unifying perspective and additional methodological contributions. The MATLAB code used in the numerical experiments is also made freely available to support the reproducibility of the results.","manuscriptTitle":"A unifying view of contrastive learning, importance sampling, and bridge sampling for energy-based models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-21 10:54:45","doi":"10.21203/rs.3.rs-9368004/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"247924365437670546312830674182503606314","date":"2026-04-16T08:27:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"169806526622628922588857918041823065789","date":"2026-04-15T13:15:28+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-04-14T04:30:59+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-04-12T00:40:57+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-04-11T12:07:09+00:00","index":"","fulltext":""},{"type":"submitted","content":"Statistics and Computing","date":"2026-04-09T11:26:22+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"statistics-and-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"stco","sideBox":"Learn more about [Statistics and Computing](http://link.springer.com/journal/11222)","snPcode":"11222","submissionUrl":"https://submission.nature.com/new-submission/11222/3","title":"Statistics and Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"78f971cc-370d-4595-948e-c4dd084590fa","owner":[],"postedDate":"April 21st, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-04-21T10:54:45+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-21 10:54:45","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9368004","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9368004","identity":"rs-9368004","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}