Maximum Entropy Sequential Design with ML-II, INLA, and MCMC Updating: A Comparative Study

Maximum Entropy Sequential Design with ML-II, INLA, and MCMC Updating: A Comparative Study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Maximum Entropy Sequential Design with ML-II, INLA, and MCMC Updating: A Comparative Study Noha Youssef This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8898682/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract This paper investigates maximum entropy sequential design for deterministic computer experiments using Gaussian process surrogates under a fixed simulation budget. We systematically compare three hyperparameter updating strategies: (i) Type-II maximum likelihood (ML-II/empirical Bayes) via marginal likelihood maximization, (ii) INLA-based approximate Bayesian updating, and (iii) maximum a posteriori (MAP) with full Bayesian propagation using MCMC. Performance is evaluated using pointwise RMSE, posterior predictive RMSE, integrated posterior variance, an entropy proxy, and computational cost. On the Forrester and Branin benchmarks, ML-II and MAP+FullBayes achieve comparable pointwise accuracy, but ML-II contracts uncertainty more aggressively, while MAP+FullBayes retains larger uncertainty due to hyperparameter propagation. INLA maintains higher integrated variance and incurs substantially greater computational cost under the present configuration. Our findings demonstrate that entropy-based sampling reliably identifies informative regions, while the updating mechanism governs the trade-off between computational efficiency and uncertainty quantification. sequential design computer experiments maximum entropy sampling integrated variance INLA marginal likelihood (ML-II) uncertainty quantification Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8898682","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":592569655,"identity":"575d9219-4036-4200-94d7-e39949616abe","order_by":0,"name":"Noha Youssef","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9klEQVRIiWNgGAWjYDAD9vYGJB4PMVp4zhyAstiI1nIjgUgt5uxnDB/z/KmT45F8/EziR0Vt4nb5BsYHb9twa7HsyTE25m07bMwjnWYm2XPmeOLONgZmw7l4tBgcyDGT5m04kLhfOsFMmrHtWOKGYwxs0rz4tJx/YyYNdFh9j+Txb9KM/8Ba2H/j1XIDaAsPG3MCjwQP0JaGGrAtzPi1PCsGuvywYQ9PTrFlz7EDxjvbEpsl55zD57DkjQ/e/KmT52E/vvHGj5o62e3Mhw9+eFOGWwsDA4cBMu+w4wYGxgZ86oGA/QEyr87eAKuqUTAKRsEoGMkAAI+6UsSdjU09AAAAAElFTkSuQmCC","orcid":"","institution":"American University in Cairo","correspondingAuthor":true,"prefix":"","firstName":"Noha","middleName":"","lastName":"Youssef","suffix":""}],"badges":[],"createdAt":"2026-02-17 07:44:26","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8898682/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8898682/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":102963781,"identity":"6d00fec1-dc7b-4229-87ee-7e3b7cc7d0f2","added_by":"auto","created_at":"2026-02-19 04:20:32","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2340849,"visible":true,"origin":"","legend":"","description":"","filename":"ModaPaper9.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8898682/v1_covered_c8e0ea3c-d79e-4662-a3a0-aef88bf0a111.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cem\u003eMaximum Entropy Sequential Design with ML-II, INLA, and MCMC Updating: A Comparative Study\u003c/em\u003e\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"The American University in Cairo","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"sequential design, computer experiments, maximum entropy sampling, integrated variance, INLA, marginal likelihood (ML-II), uncertainty quantification","lastPublishedDoi":"10.21203/rs.3.rs-8898682/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8898682/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis paper investigates maximum entropy sequential design for deterministic computer experiments using Gaussian process surrogates under a fixed simulation budget. We systematically compare three hyperparameter updating strategies: (i) Type-II maximum likelihood (ML-II/empirical Bayes) via marginal likelihood maximization, (ii) INLA-based approximate Bayesian updating, and (iii) maximum a posteriori (MAP) with full Bayesian propagation using MCMC. Performance is evaluated using pointwise RMSE, posterior predictive RMSE, integrated posterior variance, an entropy proxy, and computational cost. On the Forrester and Branin benchmarks, ML-II and MAP+FullBayes achieve comparable pointwise accuracy, but ML-II contracts uncertainty more aggressively, while MAP+FullBayes retains larger uncertainty due to hyperparameter propagation. INLA maintains higher integrated variance and incurs substantially greater computational cost under the present configuration. Our findings demonstrate that entropy-based sampling reliably identifies informative regions, while the updating mechanism governs the trade-off between computational efficiency and uncertainty quantification.\u003c/p\u003e","manuscriptTitle":"Maximum Entropy Sequential Design with ML-II, INLA, and MCMC Updating: A Comparative Study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-18 03:00:35","doi":"10.21203/rs.3.rs-8898682/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"0f7c8e0c-6e45-4565-9d35-387829f217a6","owner":[],"postedDate":"February 18th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-02-18T03:00:35+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-18 03:00:35","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8898682","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8898682","identity":"rs-8898682","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}