Variational Bayesian Inference for Modelswith Nuisance Parameters and an Intractable Likelihood

Variational Bayesian Inference for Modelswith Nuisance Parameters and an Intractable Likelihood | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Variational Bayesian Inference for Modelswith Nuisance Parameters and an Intractable Likelihood Yu-Hsiu Tseng, Minh-Ngoc Tran, Robert Kohn This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4487816/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 17 Jun, 2025 Read the published version in Statistics and Computing → Version 1 posted 9 You are reading this latest preprint version Abstract A primary challenge in Bayesian analysis lies in computing the posterior distributionof model parameters, a task that becomes more challenging for models with a largenumber of parameters or when the likelihood is intractable; as seen in Bayesian Lassoand state-space models. Often, the focus of the analysis is on a subset of the parameters,with the remainder regarded as nuisance parameters, which are either not of inferen-tial interest or are introduced to simplify computational processes. This complexitynecessitates more refined computational methods. Variational Bayesian inference (VB) has emerged as a powerful solution, enhancing computational efficiency by recastinginference as an optimization problem within a family of tractable distributions. How-ever, common VB techniques sometimes fall short, especially for models with nuisanceparameters or intractable likelihoods. To overcome these limitations, we introduce a uni-fied VB framework, termed Hybrid Variational Bayes (HVB), designed to achieve moreprecise Bayesian inference in such scenarios. This framework innovatively leverages ahybrid variational structure between the parameters of interest and nuisance parame-ters. A significant contribution of this work is the development of two robust gradientapproaches that effectively reduce variance, enhancing reliability in gradient-based op-timization algorithms for approximating complex posterior distributions. Furthermore,we link our strategy to Fisher’s identity, providing deeper theoretical insights. We alsoestablish a method to treat the stochastic element of likelihood estimation as a nuisanceparameter in models with intractable likelihoods due to complexity or computationaldemands. Through theoretical exploration and a series of illustrative examples, ourapproach demonstrates notable improvements over traditional VB methods. Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 17 Jun, 2025 Read the published version in Statistics and Computing → Version 1 posted Editorial decision: Revision requested 05 Aug, 2024 Reviews received at journal 04 Aug, 2024 Reviews received at journal 03 Jul, 2024 Reviewers agreed at journal 04 Jun, 2024 Reviewers agreed at journal 30 May, 2024 Reviewers invited by journal 30 May, 2024 Editor assigned by journal 29 May, 2024 Submission checks completed at journal 28 May, 2024 First submitted to journal 27 May, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4487816","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":310667771,"identity":"beaf183c-99b3-4da7-bf1f-8565784ae59c","order_by":0,"name":"Yu-Hsiu Tseng","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABEElEQVRIiWNgGAWjYHAD5gMMDGwQpgRh1Qkggi2BZC08BsRpMWc//PAB4486OXP+NZ8/85TdkTM4wHzwNg+DXWIDDi2WPWnGBgwJh40tZ7zdJs1z7pmxwQG2ZGsehmScWgwO5LBJMCQcSNxw4+w2Zt62w4kbDvCYSfMwMOPWcv4NSEtd/YYbZx5/Bmqp33CA/xtQSz1uLTfAtjAnGJzvYZAGakkwOMDDBtRyGLdfZgBdn5B22HDDDTYzyTnnDhvOPMxmbDnH4LgxLi3m/MkPH3ywqZM3OH/48Yc3ZYfl+Y43P7zxpqJaFqfDQEQCiJBIYGDiAdIKh+HieLSAAf8BBsYfQFoel+mjYBSMglEwYgEAwfRaB5GEVTEAAAAASUVORK5CYII=","orcid":"","institution":"The University of Sydney","correspondingAuthor":true,"prefix":"","firstName":"Yu-Hsiu","middleName":"","lastName":"Tseng","suffix":""},{"id":310667772,"identity":"ebee77fa-b15d-48ab-a2c6-5c7ca1750ae0","order_by":1,"name":"Minh-Ngoc Tran","email":"","orcid":"","institution":"The University of Sydney","correspondingAuthor":false,"prefix":"","firstName":"Minh-Ngoc","middleName":"","lastName":"Tran","suffix":""},{"id":310667773,"identity":"24c7ed06-7751-404a-af2b-9b2677b6c086","order_by":2,"name":"Robert Kohn","email":"","orcid":"","institution":"UNSW Sydney","correspondingAuthor":false,"prefix":"","firstName":"Robert","middleName":"","lastName":"Kohn","suffix":""}],"badges":[],"createdAt":"2024-05-28 03:26:57","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4487816/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4487816/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s11222-025-10654-2","type":"published","date":"2025-06-17T15:57:30+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":85231443,"identity":"aa2d5b14-7faf-4ce5-9d9a-d21e374400cd","added_by":"auto","created_at":"2025-06-23 16:08:15","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":791059,"visible":true,"origin":"","legend":"","description":"","filename":"VBfornuisanceparameters.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4487816/v1_covered_e99efe32-a39e-42b6-9fd1-e8cd5d30863e.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"\u003cp\u003eVariational Bayesian Inference for Modelswith Nuisance Parameters and an Intractable Likelihood\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"statistics-and-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"stco","sideBox":"Learn more about [Statistics and Computing](http://link.springer.com/journal/11222)","snPcode":"11222","submissionUrl":"https://submission.nature.com/new-submission/11222/3","title":"Statistics and Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4487816/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4487816/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"A primary challenge in Bayesian analysis lies in computing the posterior distributionof model parameters, a task that becomes more challenging for models with a largenumber of parameters or when the likelihood is intractable; as seen in Bayesian Lassoand state-space models. Often, the focus of the analysis is on a subset of the parameters,with the remainder regarded as nuisance parameters, which are either not of inferen-tial interest or are introduced to simplify computational processes. This complexitynecessitates more refined computational methods. Variational Bayesian inference (VB) has emerged as a powerful solution, enhancing computational efficiency by recastinginference as an optimization problem within a family of tractable distributions. How-ever, common VB techniques sometimes fall short, especially for models with nuisanceparameters or intractable likelihoods. To overcome these limitations, we introduce a uni-fied VB framework, termed Hybrid Variational Bayes (HVB), designed to achieve moreprecise Bayesian inference in such scenarios. This framework innovatively leverages ahybrid variational structure between the parameters of interest and nuisance parame-ters. A significant contribution of this work is the development of two robust gradientapproaches that effectively reduce variance, enhancing reliability in gradient-based op-timization algorithms for approximating complex posterior distributions. Furthermore,we link our strategy to Fisher’s identity, providing deeper theoretical insights. We alsoestablish a method to treat the stochastic element of likelihood estimation as a nuisanceparameter in models with intractable likelihoods due to complexity or computationaldemands. Through theoretical exploration and a series of illustrative examples, ourapproach demonstrates notable improvements over traditional VB methods.","manuscriptTitle":"Variational Bayesian Inference for Modelswith Nuisance Parameters and an Intractable Likelihood","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-10 02:21:27","doi":"10.21203/rs.3.rs-4487816/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-08-05T11:41:03+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-08-04T04:24:30+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-07-04T02:30:12+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"321403038705103341932721695589724949032","date":"2024-06-05T01:14:08+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"160840318363562703942724796918834003837","date":"2024-05-30T04:33:29+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-05-30T04:01:47+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-05-29T08:36:01+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-05-28T12:28:44+00:00","index":"","fulltext":""},{"type":"submitted","content":"Statistics and Computing","date":"2024-05-28T03:25:33+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"statistics-and-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"stco","sideBox":"Learn more about [Statistics and Computing](http://link.springer.com/journal/11222)","snPcode":"11222","submissionUrl":"https://submission.nature.com/new-submission/11222/3","title":"Statistics and Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"a99a6a4c-91cd-466b-b2c3-570adeb87b5e","owner":[],"postedDate":"June 10th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-06-23T16:04:07+00:00","versionOfRecord":{"articleIdentity":"rs-4487816","link":"https://doi.org/10.1007/s11222-025-10654-2","journal":{"identity":"statistics-and-computing","isVorOnly":false,"title":"Statistics and Computing"},"publishedOn":"2025-06-17 15:57:30","publishedOnDateReadable":"June 17th, 2025"},"versionCreatedAt":"2024-06-10 02:21:27","video":"","vorDoi":"10.1007/s11222-025-10654-2","vorDoiUrl":"https://doi.org/10.1007/s11222-025-10654-2","workflowStages":[]},"version":"v1","identity":"rs-4487816","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4487816","identity":"rs-4487816","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}