From Sampling to Structure: Constructive Optimisation for Hyperparameter Generalisation

From Sampling to Structure: Constructive Optimisation for Hyperparameter Generalisation | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article From Sampling to Structure: Constructive Optimisation for Hyperparameter Generalisation Martin Kelly This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7103211/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Traditional hyperparameter optimisation (HPO) methods operate within fixed, predefined search spaces, assuming that optimal configurations reside in statically bounded regions and may be uncovered through sampling or surrogate-guided selection. However, these methods encode inductive biases about the structure of the underlying optimisation landscape. These assumptions often fail under real-world conditions characterised by noise, heterogeneity, and evolving model architectures. This paper introduces ECO, an evolutionary cellular optimisation algorithm designed to operate under a fundamentally different premise: that search spaces themselves may be constructed, not merely interrogated. ECO models each hyperparameter as an evolving cellular lattice of alleles, dynamically expanded, refined, or pruned in response to observed fitness feedback. Through the interplay of evolutionary selection and cellular automata dynamics, ECO treats the hyperparameter space as a generative substrate, one that adapts its resolution and topology over time. We evaluate ECO across three diverse tasks, retinal optical coherence tomography (ROCT) classification, chest X-ray pathology detection, and sentiment analysis with BERT, each chosen to represent distinct structural and noise regimes. In all cases, ECO exhibits late-stage performance improvements, structural parameter sensitivity, and adaptive prioritisation of influential hyperparameters. These findings support ECO's central claim: that a generative, feedback-driven construction of the search space can yield more robust, generalisable, and context-sensitive optimisation. We position ECO not merely as an HPO algorithm, but as a representative of a new class of constructive optimisers whose adaptive structure offers a principled alternative to fixed-bias methods in modern ML. Hyperparameter optimisation Evolutionary algorithms Cellular automata Benchmark functions Adaptive search Machine learning Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7103211","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":503246620,"identity":"205e222b-72a3-48d7-8348-00b045dcd3a4","order_by":0,"name":"Martin Kelly","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA10lEQVRIiWNgGAWjYLCCBAYbBgNmBmYgg4GBjwgNjA0JDGkScC1sRGlhYDgsYcAA1MJAjBbd9rPPHzzccb7OnJ33sMHDHbXRbAzMDx/dwKPF7Ey6YUPimdsSls18yQmJZ47ntjGwGRvn4NNyII2xIbHttoTBYR7jA4ltx4BaeNik8Wo5/wyk5RwpWm6AbTkA1pKQ2FZDjJZnjDMS25IlNwC1GAD15rYxE/LL+TSGjz/b7PgNzp8xlvzZVpfbz9788DE+LejgMCx6iAd1JKofBaNgFIyCkQAAzFlL0EhlftcAAAAASUVORK5CYII=","orcid":"","institution":"thinkingML.com","correspondingAuthor":true,"prefix":"","firstName":"Martin","middleName":"","lastName":"Kelly","suffix":""}],"badges":[],"createdAt":"2025-07-11 15:53:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7103211/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7103211/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":90768124,"identity":"523f52c5-d15a-46ec-8f6e-7dad1cc68206","added_by":"auto","created_at":"2025-09-07 21:31:22","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":751097,"visible":true,"origin":"","legend":"","description":"","filename":"RXIECOMLJAuthorInfo.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7103211/v1_covered_fe9d30a9-d3b5-4418-a6ee-ae366a3cb500.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"From Sampling to Structure: Constructive Optimisation for Hyperparameter Generalisation","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Hyperparameter optimisation, Evolutionary algorithms, Cellular automata, Benchmark functions, Adaptive search, Machine learning","lastPublishedDoi":"10.21203/rs.3.rs-7103211/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7103211/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTraditional hyperparameter optimisation (HPO) methods operate within fixed, predefined search spaces, assuming that optimal configurations reside in statically bounded regions and may be uncovered through sampling or surrogate-guided selection. However, these methods encode inductive biases about the structure of the underlying optimisation landscape. These assumptions often fail under real-world conditions characterised by noise, heterogeneity, and evolving model architectures. This paper introduces ECO, an evolutionary cellular optimisation algorithm designed to operate under a fundamentally different premise: that search spaces themselves may be constructed, not merely interrogated.\u003c/p\u003e\u003cp\u003eECO models each hyperparameter as an evolving cellular lattice of alleles, dynamically expanded, refined, or pruned in response to observed fitness feedback. Through the interplay of evolutionary selection and cellular automata dynamics, ECO treats the hyperparameter space as a generative substrate, one that adapts its resolution and topology over time. We evaluate ECO across three diverse tasks, retinal optical coherence tomography (ROCT) classification, chest X-ray pathology detection, and sentiment analysis with BERT, each chosen to represent distinct structural and noise regimes. In all cases, ECO exhibits late-stage performance improvements, structural parameter sensitivity, and adaptive prioritisation of influential hyperparameters.\u003c/p\u003e\u003cp\u003eThese findings support ECO's central claim: that a generative, feedback-driven construction of the search space can yield more robust, generalisable, and context-sensitive optimisation. We position ECO not merely as an HPO algorithm, but as a representative of a new class of constructive optimisers whose adaptive structure offers a principled alternative to fixed-bias methods in modern ML.\u003c/p\u003e","manuscriptTitle":"From Sampling to Structure: Constructive Optimisation for Hyperparameter Generalisation","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-08-29 13:41:22","doi":"10.21203/rs.3.rs-7103211/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"0a274c94-eacc-4cd3-9d5d-da8a69523602","owner":[],"postedDate":"August 29th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-09-07T21:23:12+00:00","versionOfRecord":[],"versionCreatedAt":"2025-08-29 13:41:22","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7103211","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7103211","identity":"rs-7103211","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}