Adaptive Mesh Refinement with Dynamic Load Balancing for Scalable Cosmological Simulations: A Hybrid Meta-Heuristic Approach

Adaptive Mesh Refinement with Dynamic Load Balancing for Scalable Cosmological Simulations: A Hybrid Meta-Heuristic Approach | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Adaptive Mesh Refinement with Dynamic Load Balancing for Scalable Cosmological Simulations: A Hybrid Meta-Heuristic Approach xiaochen xiao This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6501625/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract We propose a novel framework for cosmological simulations that integrates adaptive mesh refinement (AMR) with dynamic load balancing to address the challenges of spatial-temporal complexity in high-performance computing (HPC) environments. The proposed system employs a hybrid meta-heuristic algorithm, combining particle-swarm optimization with gradient-based heuristics, to dynamically redistribute computational workloads while minimizing communication overhead and latency penalties. The AMR component utilizes a wavelet-based error estimator to adaptively adjust mesh resolution, ensuring physical consistency with refinement criteria tied to local density gradients. Furthermore, the framework replaces conventional static domain decomposition with a dynamic task redistribution mechanism, which interacts seamlessly with gravity and hydrodynamics solvers through non-blocking MPI-3.0 interfaces. Implemented on GPU-accelerated HPC clusters, the system extends the AMReX library with custom CUDA kernels and integrates a distributed Rust-based load balancer for low-latency task migration. The co-design of AMR and load balancing eliminates traditional synchronization bottlenecks, achieving a 30--40% reduction in preliminary benchmarks. This approach significantly enhances scalability and resolution fidelity in large-scale cosmological simulations, offering a robust solution for modern computational astrophysics. The novelty lies in the tight coupling of mesh adaptation and workload optimization, which collectively improve efficiency without compromising accuracy. Physical sciences/Astronomy and planetary science/Astronomy and astrophysics/Stars Physical sciences/Astronomy and planetary science/Astronomy and astrophysics/Time-domain astronomy Adaptive mesh refinement dynamic load balancing cosmological simulations high-performance computing meta-heuristics Full Text Additional Declarations There is NO Competing Interest. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6501625","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":446192191,"identity":"961f29d7-af9e-465e-9abe-80cb77b6a98c","order_by":0,"name":"xiaochen xiao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAqklEQVRIiWNgGAWjYHAC9h8fDCTkSNMjOaPAwpg0LdI8HyoSG4hWbj4jgcFwhoFEet/xBMYPH3OI0CJz5gBDAtAvuTPPPGCWnLmNCC0S7A0MB4G25G64kcDGzEuUFmYGxmYeoMMMiNfC3sDMDNSSQIIWngNsjECHGc4887CZSL9IJLAxfPhTJ893PPngh4/EaGFg4P8AoQ+QEDVQcCCBVB2jYBSMglEwUgAA1ZMycYH6NxYAAAAASUVORK5CYII=","orcid":"https://orcid.org/0009-0000-9117-1080","institution":"University of Technology Sydney","correspondingAuthor":true,"prefix":"","firstName":"xiaochen","middleName":"","lastName":"xiao","suffix":""}],"badges":[],"createdAt":"2025-04-22 07:51:37","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6501625/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6501625/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":83286617,"identity":"8322cb7d-5846-42e3-9701-333552bd182a","added_by":"auto","created_at":"2025-05-22 11:39:45","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":907760,"visible":true,"origin":"","legend":"Article File","description":"","filename":"UNIVERISE01.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6501625/v1_covered_572e9872-6e83-4208-9240-a2f6a50c5724.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Adaptive Mesh Refinement with Dynamic Load Balancing for Scalable Cosmological Simulations: A Hybrid Meta-Heuristic Approach","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Adaptive mesh refinement, dynamic load balancing, cosmological simulations, high-performance computing, meta-heuristics","lastPublishedDoi":"10.21203/rs.3.rs-6501625/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6501625/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"We propose a novel framework for cosmological simulations that integrates adaptive mesh refinement (AMR) with dynamic load balancing to address the challenges of spatial-temporal complexity in high-performance computing (HPC) environments. The proposed system employs a hybrid meta-heuristic algorithm, combining particle-swarm optimization with gradient-based heuristics, to dynamically redistribute computational workloads while minimizing communication overhead and latency penalties. The AMR component utilizes a wavelet-based error estimator to adaptively adjust mesh resolution, ensuring physical consistency with refinement criteria tied to local density gradients. Furthermore, the framework replaces conventional static domain decomposition with a dynamic task redistribution mechanism, which interacts seamlessly with gravity and hydrodynamics solvers through non-blocking MPI-3.0 interfaces. Implemented on GPU-accelerated HPC clusters, the system extends the AMReX library with custom CUDA kernels and integrates a distributed Rust-based load balancer for low-latency task migration. The co-design of AMR and load balancing eliminates traditional synchronization bottlenecks, achieving a 30--40\\% reduction in preliminary benchmarks. This approach significantly enhances scalability and resolution fidelity in large-scale cosmological simulations, offering a robust solution for modern computational astrophysics. The novelty lies in the tight coupling of mesh adaptation and workload optimization, which collectively improve efficiency without compromising accuracy.","manuscriptTitle":"Adaptive Mesh Refinement with Dynamic Load Balancing for Scalable Cosmological Simulations: A Hybrid Meta-Heuristic Approach","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-23 03:55:58","doi":"10.21203/rs.3.rs-6501625/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"652dbf19-4683-48c4-b2d8-dcc4bb7ea119","owner":[],"postedDate":"April 23rd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":47491811,"name":"Physical sciences/Astronomy and planetary science/Astronomy and astrophysics/Stars"},{"id":47491812,"name":"Physical sciences/Astronomy and planetary science/Astronomy and astrophysics/Time-domain astronomy"}],"tags":[],"updatedAt":"2025-05-22T13:05:57+00:00","versionOfRecord":[],"versionCreatedAt":"2025-04-23 03:55:58","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6501625","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6501625","identity":"rs-6501625","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}