Decreased Max-value Entropy Search for Multi-fidelity Bayesian Optimization

Decreased Max-value Entropy Search for Multi-fidelity Bayesian Optimization | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Decreased Max-value Entropy Search for Multi-fidelity Bayesian Optimization Zhuo Gong, Binglin Wang, Xiaojun Duan, Jiangtao Chen, Liang Yan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3118480/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Bayesian Optimization is a widely applied efficient framework for updating the surrogate model sequentially. To improve the efficiency, multi-fidelity Bayesian Optimization is developed to combine the information of samples in different fidelity levels. However, the multi-fidelity levels brings the challenge for sequential sampling. In multi-fidelity Bayesian Optimization, the sampling strategy is applied to determine not only the sample location but also the sample fidelity level for updating the model. To balance the benefit and the experiment cost, it is vital to measure the potential effect of each fidelity sample. Some sampling strategies, which is categorized as direct-type methods, can measure the potential effect of different samples in a easy way, but they cannot figure out the difference of samples that has little uncertainty and it has the risk for redundant sampling. Some other strategies, which is categorized as direct-type methods, can measure the potential effect appropriately, but the calculation of them are much complicated and time-consuming. In this paper, a new type method combining the direct-type and indirect-type methods is presented for measuring the potential effect of different fidelity sample. It is convenient enough and can avoid the problems occurring in the direct-type method. Based on this paradigm, a sampling strategy named decreased max-value entropy search(DMES) is proposed and applied in the multi-fidelity Bayesian Optimization framework. The characteristics of DMES and how it is different from direct-type method are detailed in some examples. Besides, two numerical experiments and one simulation experiment demonstrate the efficiency of DMES. Physical sciences/Mathematics and computing/Applied mathematics Physical sciences/Mathematics and computing/Computer science Physical sciences/Mathematics and computing/Information technology Physical sciences/Mathematics and computing/Statistics Full Text Additional Declarations No competing interests reported. Supplementary Files Appendix.pdf Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3118480","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":220181357,"identity":"658dc03f-b5e2-44c3-b4b5-7e724bb83e6d","order_by":0,"name":"Zhuo Gong","email":"","orcid":"","institution":"College of Science, National University of Defense Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Zhuo","middleName":"","lastName":"Gong","suffix":""},{"id":220181358,"identity":"3af93d3d-22da-4e91-a2ee-702e62ef3679","order_by":1,"name":"Binglin Wang","email":"","orcid":"","institution":"College of Science, National University of Defense Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Binglin","middleName":"","lastName":"Wang","suffix":""},{"id":220181359,"identity":"9451a2a8-0502-4730-a51b-a1d6be79768e","order_by":2,"name":"Xiaojun Duan","email":"","orcid":"","institution":"College of Science, National University of Defense Technology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Xiaojun","middleName":"","lastName":"Duan","suffix":""},{"id":220181360,"identity":"0250ac30-70e3-4b61-9129-e4a370172d45","order_by":3,"name":"Jiangtao Chen","email":"","orcid":"","institution":"China Aerodynamics Research and Development Center","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Jiangtao","middleName":"","lastName":"Chen","suffix":""},{"id":220181361,"identity":"30961335-e6e3-4abd-a5f5-5a652a624fbc","order_by":4,"name":"Liang Yan","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA50lEQVRIiWNgGAWjYHACxgMJQJIfiCVAvAZi9IC1SAKVShwgWguIMDhArBaD42cPHHhQc8du8+0ew9sfGGxkNxxgfvYAr5YzeQkHEo49S95254yxxQGGNOMNB9jMDfBqOZBjcCCB7XCy2Y0cM6DDDiduOMDDJoFXy/k3QC3/DicbzwBr+U+ElhtAWxLbDtsZSIC1HCCsRfIG0JbEvsMJEjfSii3OGCQbzzzMZoZXC9/5HMOHP74dtuefkbzxRkWFnWzf8eZneLUoHIDQiQ0QdwIxMz71QCAPUcpgT0DdKBgFo2AUjGQAAOsMVbeRWsjYAAAAAElFTkSuQmCC","orcid":"","institution":"College of Science, National University of Defense Technology","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Liang","middleName":"","lastName":"Yan","suffix":""}],"badges":[],"createdAt":"2023-06-28 05:29:25","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3118480/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3118480/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":41157117,"identity":"5bf0c7cb-e3f8-426d-b9d5-4bf8cb97d004","added_by":"auto","created_at":"2023-08-07 06:37:36","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1722075,"visible":true,"origin":"","legend":"","description":"","filename":"DecreasedMaxvalueEntropySearchforMultifidelityBayesianOptimization.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3118480/v1_covered_9987745c-30bd-4749-8ef9-3bd05c78506d.pdf"},{"id":40492749,"identity":"04fd4a89-5cde-47f4-bba2-bd6d01c9df9b","added_by":"auto","created_at":"2023-07-24 19:04:21","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":88775,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3118480/v1/7a24ff116a24ecae67d9a439.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Decreased Max-value Entropy Search for Multi-fidelity Bayesian Optimization","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-3118480/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3118480/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Bayesian Optimization is a widely applied efficient framework for updating the surrogate model sequentially. To improve the efficiency, multi-fidelity Bayesian Optimization is developed to combine the information of samples in different fidelity levels. However, the multi-fidelity levels brings the challenge for sequential sampling. In multi-fidelity Bayesian Optimization, the sampling strategy is applied to determine not only the sample location but also the sample fidelity level for updating the model. To balance the benefit and the experiment cost, it is vital to measure the potential effect of each fidelity sample. Some sampling strategies, which is categorized as direct-type methods, can measure the potential effect of different samples in a easy way, but they cannot figure out the difference of samples that has little uncertainty and it has the risk for redundant sampling. Some other strategies, which is categorized as direct-type methods, can measure the potential effect appropriately, but the calculation of them are much complicated and time-consuming. In this paper, a new type method combining the direct-type and indirect-type methods is presented for measuring the potential effect of different fidelity sample. It is convenient enough and can avoid the problems occurring in the direct-type method. Based on this paradigm, a sampling strategy named decreased max-value entropy search(DMES) is proposed and applied in the multi-fidelity Bayesian Optimization framework. The characteristics of DMES and how it is different from direct-type method are detailed in some examples. Besides, two numerical experiments and one simulation experiment demonstrate the efficiency of DMES.","manuscriptTitle":"Decreased Max-value Entropy Search for Multi-fidelity Bayesian Optimization","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2023-07-24 19:04:16","doi":"10.21203/rs.3.rs-3118480/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"53696690-f6b2-4140-abf7-5430346a1bf3","owner":[],"postedDate":"July 24th, 2023","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":23419177,"name":"Physical sciences/Mathematics and computing/Applied mathematics"},{"id":23419178,"name":"Physical sciences/Mathematics and computing/Computer science"},{"id":23419179,"name":"Physical sciences/Mathematics and computing/Information technology"},{"id":23419180,"name":"Physical sciences/Mathematics and computing/Statistics"}],"tags":[],"updatedAt":"2023-08-07T06:29:21+00:00","versionOfRecord":[],"versionCreatedAt":"2023-07-24 19:04:16","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3118480","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3118480","identity":"rs-3118480","version":["v1"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}