Parameter Expanded Variational Bayes for Well-Calibrated High-Dimensional Linear Regression with Spike-and-Slab Priors

Parameter Expanded Variational Bayes for Well-Calibrated High-Dimensional Linear Regression with Spike-and-Slab Priors | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Parameter Expanded Variational Bayes for Well-Calibrated High-Dimensional Linear Regression with Spike-and-Slab Priors Peter Olejua, Enakshi Saha, Rahul Ghosal, Ray Bai, Alexander McLain This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7208847/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 9 You are reading this latest preprint version Abstract As scientific problems grow in complexity, there is a pressing need for robust and scalable computational methods for fitting high-dimensional statistical models. Variational Bayes (VB) provides an approximate alternative to traditional sampling-based Bayesian inference, often reducing computation time from days to hours or minutes. VB typically minimizes the Kullback-Leibler divergence via coordinate ascent under a mean-field assumption. Its performance can be highly sensitive to prior specifications, particularly in sparse high-dimensional regression with spike-and-slab priors. A significant limitation of standard VB is its tendency to produce poorly calibrated predictions; that is, the predicted values often exhibit a systematic bias relative to the observed outcomes, failing to accurately reflect the true conditional expectation. Motivated by this, we apply parameter expansion to VB and propose a sparse parameter-expanded VB (spexvb) algorithm that improves robustness to prior settings and enhances predictive calibration. Compared to standard VB, spexvb demonstrates significantly enhanced robustness to prior specifications, yielding consistently lower predictive error, improved variable selection accuracy, and more stable and accurate posterior estimates, particularly in sparse and high-dimensional settings. We evaluate its performance through extensive simulations and a real-world application, demonstrating the practical advantages of parameter expansion in variational inference for high-dimensional regression. Variational Bayes Parameter-Expanded Variational Bayes High-dimensional Variable Selection Spike-and-Slab Priors Full Text Additional Declarations No competing interests reported. Supplementary Files supplementary.pdf Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 25 Oct, 2025 Reviews received at journal 19 Oct, 2025 Reviews received at journal 29 Aug, 2025 Reviewers agreed at journal 17 Aug, 2025 Reviewers agreed at journal 13 Aug, 2025 Reviewers invited by journal 30 Jul, 2025 Editor assigned by journal 27 Jul, 2025 Submission checks completed at journal 26 Jul, 2025 First submitted to journal 24 Jul, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7208847","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":493315453,"identity":"2a40ba0f-ab9d-4038-8589-6550902e9486","order_by":0,"name":"Peter Olejua","email":"","orcid":"","institution":"University of South Carolina","correspondingAuthor":false,"prefix":"","firstName":"Peter","middleName":"","lastName":"Olejua","suffix":""},{"id":493315460,"identity":"ffff88f1-f03a-484b-99b1-ea126f5cce13","order_by":1,"name":"Enakshi Saha","email":"","orcid":"","institution":"University of South Carolina","correspondingAuthor":false,"prefix":"","firstName":"Enakshi","middleName":"","lastName":"Saha","suffix":""},{"id":493315462,"identity":"13689c10-5c67-43d2-bbbe-21e5f9159847","order_by":2,"name":"Rahul Ghosal","email":"","orcid":"","institution":"University of South Carolina","correspondingAuthor":false,"prefix":"","firstName":"Rahul","middleName":"","lastName":"Ghosal","suffix":""},{"id":493315464,"identity":"4a6e7909-f4c9-48b9-b8c4-bb1a1c23bff1","order_by":3,"name":"Ray Bai","email":"","orcid":"","institution":"George Mason University","correspondingAuthor":false,"prefix":"","firstName":"Ray","middleName":"","lastName":"Bai","suffix":""},{"id":493315465,"identity":"d0f82a76-342d-4275-8f87-afe8e8ec026c","order_by":4,"name":"Alexander McLain","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA50lEQVRIiWNgGAWjYFAC5gaGBAYGOQYGxsbDcEEevFoYwVqMQQwStABBIogkTotu+8HGDw932KWvbT/ccLig4jCDfPsBxgdv23BrMTuT2CyReCY5d9uZxIbDM86kMRicSWA2nItPyw3GBonENubcbQeAWnjbbBgMJBjYpHnxa2n+kdhWn252/iFIiwSD/AwG9t8EtLQBbTmcYHYDagvDDQY2ZrxaziS2WSS2HTfcduMh2C88BkDfSc45h0fL8cOHb/5sq5Y3O5/+8DEwxOTk2w8f/PCmDLcWDMADi6lRMApGwSgYBRQAAD/7V7H9Yq+HAAAAAElFTkSuQmCC","orcid":"","institution":"University of South Carolina","correspondingAuthor":true,"prefix":"","firstName":"Alexander","middleName":"","lastName":"McLain","suffix":""}],"badges":[],"createdAt":"2025-07-24 22:08:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7208847/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7208847/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":88341351,"identity":"d5745c7e-cc2a-4d7c-8d5a-1ae0bd8a485d","added_by":"auto","created_at":"2025-08-05 12:42:53","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":677105,"visible":true,"origin":"","legend":"","description":"","filename":"spexvbSubmission.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7208847/v1_covered_93c1d413-4dfe-4a45-8781-36223a5f512c.pdf"},{"id":88340108,"identity":"fe5446fe-39ab-459d-866c-f2bfc0efe598","added_by":"auto","created_at":"2025-08-05 12:34:52","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":531155,"visible":true,"origin":"","legend":"","description":"","filename":"supplementary.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7208847/v1/37b5b3edcb0a50af819b8f73.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Parameter Expanded Variational Bayes for Well-Calibrated High-Dimensional Linear Regression with Spike-and-Slab Priors","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"statistics-and-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"stco","sideBox":"Learn more about [Statistics and Computing](http://link.springer.com/journal/11222)","snPcode":"11222","submissionUrl":"https://submission.nature.com/new-submission/11222/3","title":"Statistics and Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Variational Bayes, Parameter-Expanded Variational Bayes, High-dimensional Variable Selection, Spike-and-Slab Priors","lastPublishedDoi":"10.21203/rs.3.rs-7208847/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7208847/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"As scientific problems grow in complexity, there is a pressing need for robust and scalable computational methods for fitting high-dimensional statistical models. Variational Bayes (VB) provides an approximate alternative to traditional sampling-based Bayesian inference, often reducing computation time from days to hours or minutes. VB typically minimizes the Kullback-Leibler divergence via coordinate ascent under a mean-field assumption. Its performance can be highly sensitive to prior specifications, particularly in sparse high-dimensional regression with spike-and-slab priors. A significant limitation of standard VB is its tendency to produce poorly calibrated predictions; that is, the predicted values often exhibit a systematic bias relative to the observed outcomes, failing to accurately reflect the true conditional expectation. Motivated by this, we apply parameter expansion to VB and propose a sparse parameter-expanded VB (spexvb) algorithm that improves robustness to prior settings and enhances predictive calibration. Compared to standard VB, spexvb demonstrates significantly enhanced robustness to prior specifications, yielding consistently lower predictive error, improved variable selection accuracy, and more stable and accurate posterior estimates, particularly in sparse and high-dimensional settings. We evaluate its performance through extensive simulations and a real-world application, demonstrating the practical advantages of parameter expansion in variational inference for high-dimensional regression.","manuscriptTitle":"Parameter Expanded Variational Bayes for Well-Calibrated High-Dimensional Linear Regression with Spike-and-Slab Priors","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-08-05 12:34:47","doi":"10.21203/rs.3.rs-7208847/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-10-25T08:40:09+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-10-19T22:53:18+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-29T15:39:05+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"247619816437423486224648264140236579115","date":"2025-08-17T16:22:53+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"254960854130883372658978940520286445449","date":"2025-08-13T14:55:15+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-07-30T16:35:47+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-07-27T05:38:19+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-07-26T10:18:34+00:00","index":"","fulltext":""},{"type":"submitted","content":"Statistics and Computing","date":"2025-07-24T21:55:34+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"statistics-and-computing","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"stco","sideBox":"Learn more about [Statistics and Computing](http://link.springer.com/journal/11222)","snPcode":"11222","submissionUrl":"https://submission.nature.com/new-submission/11222/3","title":"Statistics and Computing","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"a5f32c6f-5df2-4cb8-aca8-c44dc6779cd2","owner":[],"postedDate":"August 5th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-01-26T12:23:22+00:00","versionOfRecord":[],"versionCreatedAt":"2025-08-05 12:34:47","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7208847","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7208847","identity":"rs-7208847","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}