Entropy Change in Sampling Predicts Downstream Performance in Neural Networks

Entropy Change in Sampling Predicts Downstream Performance in Neural Networks | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Entropy Change in Sampling Predicts Downstream Performance in Neural Networks Jun-Jie Zhang, Xiu-Cheng Wang, Nan Cheng, Long-Gang Pang, Tai-Jiao Du, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8034605/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Data acquisition, the physical interface between the world and learning systems, fundamentally determines how much information is available before any model is trained. Existing evaluation and optimization methods typically rely on learned losses or domain-specific heuristics. Here, we introduce a training-free scalar—band-entropy change—which quantifies how an acquisition process disturbs the spectral structure of the signal. Across three domains—vision (patch masking), wireless multiple-input multiple-output (MIMO) systems (pilot/antenna subsampling), and magnetic resonance imaging (MRI) ($k$-space undersampling$)$—we show experimentally that the magnitude of band-entropy change, computed from raw measurements, provides a useful, training-free indicator of downstream performance: smaller values are consistently associated with higher classification accuracy or reconstruction quality. Our results motivate a task-agnostic, instrument-aware diagnostic—entropy auditing—that can evaluate or guide acquisition choices prior to training. This framework bridges the gap between physical measurement principles and machine learning, offering a physics-informed approach to optimizing sampling strategies and predicting downstream performance across various domains. Physical sciences/Mathematics and computing/Applied mathematics Physical sciences/Mathematics and computing/Computational science Physical sciences/Physics/Information theory and computation entropy auditing Husimi spectrogram subsampling acquisition design vision MIMO MRI Full Text Additional Declarations There is NO Competing Interest. Supplementary Files SI.pdf Supplementary Information for Entropy Change in Sampling Predicts Downstream Performance in Neural Networks Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8034605","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":540639805,"identity":"04847dd9-8fcb-491c-ba57-72454c287e34","order_by":0,"name":"Jun-Jie Zhang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA+ElEQVRIiWNgGAWjYFCCAwwMjA02DAzMzA0wIQNitKQBtTA2wFQT0sIA0nIYTBKnRb7x8MMPH3ecj+ZvZ2xg5vnzR56BvXmbBEPNHTwWHDOWnHnmdu6Mw0AtvG0Ghg08x8okGI49w6mFmeGAGVDl7dwGsJYGgwQGiRwzCYhTsQM2huPfgFrO5c4/DHYYUIv8G/xaeBjOgGw5kLsBrIUNZAsPfi0SDGeKgX5Jzt0I1HJwbpuxYRtPWrFFwjHcWuRnHN8IDDG73HnnDx988OaPnDw/++GNNz7U4NbCIHEAwT4A8R0QJODWwMDA34BPdhSMglEwCkYBEAAAyWlUDIaxrtMAAAAASUVORK5CYII=","orcid":"","institution":"Northwest Institute of Nuclear Technology","correspondingAuthor":true,"prefix":"","firstName":"Jun-Jie","middleName":"","lastName":"Zhang","suffix":""},{"id":540639806,"identity":"2135cd68-e0be-4979-bae7-e55ef4a08db6","order_by":1,"name":"Xiu-Cheng Wang","email":"","orcid":"","institution":"Xidian University","correspondingAuthor":false,"prefix":"","firstName":"Xiu-Cheng","middleName":"","lastName":"Wang","suffix":""},{"id":540639807,"identity":"0d5ab2c4-3ed3-4a1c-bfc9-417aac324697","order_by":2,"name":"Nan Cheng","email":"","orcid":"","institution":"Xidian University","correspondingAuthor":false,"prefix":"","firstName":"Nan","middleName":"","lastName":"Cheng","suffix":""},{"id":540639808,"identity":"6dd961d1-07fd-4190-b62e-32532f31269d","order_by":3,"name":"Long-Gang Pang","email":"","orcid":"","institution":"Central China Normal University","correspondingAuthor":false,"prefix":"","firstName":"Long-Gang","middleName":"","lastName":"Pang","suffix":""},{"id":540639809,"identity":"856c60b9-63de-4f53-8c3d-9a9b38b5bc29","order_by":4,"name":"Tai-Jiao Du","email":"","orcid":"","institution":"Northwest Institute of Nuclear Technology","correspondingAuthor":false,"prefix":"","firstName":"Tai-Jiao","middleName":"","lastName":"Du","suffix":""},{"id":540639810,"identity":"7ba354ca-8e43-4aaf-8752-e9a118ba7571","order_by":5,"name":"Deyu Meng","email":"","orcid":"","institution":"Xi’an Jiaotong University","correspondingAuthor":false,"prefix":"","firstName":"Deyu","middleName":"","lastName":"Meng","suffix":""}],"badges":[],"createdAt":"2025-11-05 06:05:48","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8034605/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8034605/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":96455539,"identity":"840f0c67-0d82-4c30-aec1-a31659468b9e","added_by":"auto","created_at":"2025-11-21 10:04:19","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":7030076,"visible":true,"origin":"","legend":"Article File","description":"","filename":"main.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8034605/v1_covered_3397139a-279c-4532-8e49-5e89eb815f98.pdf"},{"id":96130463,"identity":"d176bf9a-6a34-4436-ba03-c82a30057cbb","added_by":"auto","created_at":"2025-11-18 02:13:45","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":1058562,"visible":true,"origin":"","legend":"Supplementary Information for Entropy Change in Sampling Predicts Downstream Performance in Neural Networks","description":"","filename":"SI.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8034605/v1/ecb1f19555a4ddcb6dbfc26c.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"Entropy Change in Sampling Predicts Downstream Performance in Neural Networks","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"entropy auditing, Husimi spectrogram, subsampling, acquisition design, vision, MIMO, MRI","lastPublishedDoi":"10.21203/rs.3.rs-8034605/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8034605/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Data acquisition, the physical interface between the world and learning systems, fundamentally determines how much information is available before any model is trained. Existing evaluation and optimization methods typically rely on learned losses or domain-specific heuristics. Here, we introduce a training-free scalar—band-entropy change—which quantifies how an acquisition process disturbs the spectral structure of the signal. \r\nAcross three domains—vision (patch masking), wireless multiple-input multiple-output (MIMO) systems (pilot/antenna subsampling), and magnetic resonance imaging (MRI) ($k$-space undersampling$)$—we show experimentally that the magnitude of band-entropy change, computed from raw measurements, provides a useful, training-free indicator of downstream performance: smaller values are consistently associated with higher classification accuracy or reconstruction quality.\r\nOur results motivate a task-agnostic, instrument-aware diagnostic—entropy auditing—that can evaluate or guide acquisition choices prior to training. This framework bridges the gap between physical measurement principles and machine learning, offering a physics-informed approach to optimizing sampling strategies and predicting downstream performance across various domains.","manuscriptTitle":"Entropy Change in Sampling Predicts Downstream Performance in Neural Networks","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-11-18 02:13:40","doi":"10.21203/rs.3.rs-8034605/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"a32a9abe-7c51-4db3-8ad0-262cc33c7c8f","owner":[],"postedDate":"November 18th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":57513400,"name":"Physical sciences/Mathematics and computing/Applied mathematics"},{"id":57513401,"name":"Physical sciences/Mathematics and computing/Computational science"},{"id":57513402,"name":"Physical sciences/Physics/Information theory and computation"}],"tags":[],"updatedAt":"2025-11-21T12:10:35+00:00","versionOfRecord":[],"versionCreatedAt":"2025-11-18 02:13:40","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8034605","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8034605","identity":"rs-8034605","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}