A brain-rhythm hierarchical predictive computations integrate semantics and acoustics in speech processing

A brain-rhythm hierarchical predictive computations integrate semantics and acoustics in speech processing | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article A brain-rhythm hierarchical predictive computations integrate semantics and acoustics in speech processing Olesia Dogonasheva, Keith Doelling, Denis Zakharov, Anne-Lise Giraud, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4436764/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 26 Sep, 2025 Read the published version in Nature Computational Science → Version 1 posted You are reading this latest preprint version Abstract Unraveling how humans effortlessly grasp speech despite diverse environmental challenges has long intrigued researchers in systems and cognitive neuroscience. The interplay between semantic and phonological language structures has been a subject of debate in the linguistics and neurolinguistics literature that, so far, has not been resolved. We seek to understand the neural intricacies underpinning semantic-acoustic interplay in robust speech comprehension. To do so, we construct a computational mechanistic proof for the hypothesis, proposing a pivotal role for rhythmic predictive top-down contextualization facilitated by the delta rhythm in achieving time-invariant speech processing. Our Brain-Rhythm-based Inference model, BRyBI, integrates three key rhythmic processes -- theta-gamma interactions for parsing phoneme sequences, dynamic delta rhythm for inferred prosodic-phrase context, and resilient speech representations. Demonstrating mechanistic proof-of-principle, BRyBI replicates human behavioral experiments, showcasing its ability to handle pitch variations, time-warped speech, interruptions, and silences in non-comprehensible contexts. Intriguingly, the model aligns with human experiments, revealing optimal silence time scales in the theta- and delta-frequency ranges. Comparative analysis with deep neural network language models highlights distinctive performance patterns, emphasizing the unique capabilities of a rhythmic framework. In essence, our study sheds light on the neural underpinnings of speech processing, emphasizing the role of rhythmic brain mechanisms in structured temporal signal processing -- an insight that challenges prevailing artificial intelligence paradigms and hints at potential advancements in compact and robust computing architectures. Biological sciences/Neuroscience/Computational neuroscience/Dynamical systems Biological sciences/Neuroscience/Computational neuroscience/Neural encoding rhythms predictive coding speech recognition inference model invariant speech processing auditory cortex Full Text Additional Declarations There is NO Competing Interest. Supplementary Files BRyBIpapersuppl.pdf Cite Share Download PDF Status: Published Journal Publication published 26 Sep, 2025 Read the published version in Nature Computational Science → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4436764","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":315204506,"identity":"6fb2fde4-cb55-4a14-b8db-eaa43ca7b37a","order_by":0,"name":"Olesia Dogonasheva","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6klEQVRIiWNgGAWjYFCCBAhlACI+ADEbOylaGGeAtDCTooWZB8QipIWfPfnwi4877OzN2XsMP9v82ibPx8zA+OFjDm4tkj3P0ixnnklO3Nlzxlg6t++2YRszA7PkzG24tRjcyDEz5m1jTjC4kZYgndtzmxGohY2Zl7CWenuD+8+Sf1v23LYnRovxY962w4wbbjAfk2b4cTuRoBaQXxhnth1P3HAm+Zhlb8Pt5DZmxma8fgGF2IePbdX2BscPNt/48ee27fz25oMfPuLRAgRsEnAmYxuYbMCrHgiYPyDYfwgpHgWjYBSMgpEIAHoSUosYGS8uAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0003-2053-1661","institution":"Institut de l'Audition, The Institut Pasteur, Université de Paris Cité","correspondingAuthor":true,"prefix":"","firstName":"Olesia","middleName":"","lastName":"Dogonasheva","suffix":""},{"id":315204507,"identity":"e823a713-3e3c-43ca-8a31-c596695a2d44","order_by":1,"name":"Keith Doelling","email":"","orcid":"","institution":"Institut de l'Audition, The Institut Pasteur, Université de Paris Cité","correspondingAuthor":false,"prefix":"","firstName":"Keith","middleName":"","lastName":"Doelling","suffix":""},{"id":315204508,"identity":"0c1c8ea7-51bf-4677-93c5-cbb3165170c5","order_by":2,"name":"Denis Zakharov","email":"","orcid":"","institution":"Higher School of Economics","correspondingAuthor":false,"prefix":"","firstName":"Denis","middleName":"","lastName":"Zakharov","suffix":""},{"id":315204509,"identity":"f6b99ae7-0aee-4570-b745-a2e5c285912a","order_by":3,"name":"Anne-Lise Giraud","email":"","orcid":"","institution":"Institut de l'Audition, The Institut Pasteur, Université de Paris Cité","correspondingAuthor":false,"prefix":"","firstName":"Anne-Lise","middleName":"","lastName":"Giraud","suffix":""},{"id":315204510,"identity":"e373452f-01ac-44d3-9bbf-4eac8b37c3f4","order_by":4,"name":"Boris Gutkin","email":"","orcid":"","institution":"Ecole Normale Supérieure","correspondingAuthor":false,"prefix":"","firstName":"Boris","middleName":"","lastName":"Gutkin","suffix":""}],"badges":[],"createdAt":"2024-05-17 12:35:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4436764/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4436764/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s43588-025-00876-9","type":"published","date":"2025-09-26T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":92304518,"identity":"80fa35fa-ab2c-445d-aaf2-4f98f76050e5","added_by":"auto","created_at":"2025-09-27 07:07:52","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3290960,"visible":true,"origin":"","legend":"","description":"","filename":"BRyBIpaperMAI2.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4436764/v1_covered_34abe638-6e17-4c15-868e-ba8b4affd996.pdf"},{"id":58533651,"identity":"cbd3e5b2-dc63-49fe-b971-49583c165d11","added_by":"auto","created_at":"2024-06-18 01:50:28","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":6340603,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"BRyBIpapersuppl.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4436764/v1/ef4a7cc1399fed75a54d1b7b.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"A brain-rhythm hierarchical predictive computations integrate semantics and acoustics in speech processing","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"rhythms, predictive coding, speech recognition, inference model, invariant speech processing, auditory cortex","lastPublishedDoi":"10.21203/rs.3.rs-4436764/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4436764/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Unraveling how humans effortlessly grasp speech despite diverse environmental challenges has long intrigued researchers in systems and cognitive neuroscience. The interplay between semantic and phonological language structures has been a subject of debate in the linguistics and neurolinguistics literature that, so far, has not been resolved. We seek to understand the neural intricacies underpinning semantic-acoustic interplay in robust speech comprehension. To do so, we construct a computational mechanistic proof for the hypothesis, proposing a pivotal role for rhythmic predictive top-down contextualization facilitated by the delta rhythm in achieving time-invariant speech processing. Our Brain-Rhythm-based Inference model, BRyBI, integrates three key rhythmic processes -- theta-gamma interactions for parsing phoneme sequences, dynamic delta rhythm for inferred prosodic-phrase context, and resilient speech representations. Demonstrating mechanistic proof-of-principle, BRyBI replicates human behavioral experiments, showcasing its ability to handle pitch variations, time-warped speech, interruptions, and silences in non-comprehensible contexts. Intriguingly, the model aligns with human experiments, revealing optimal silence time scales in the theta- and delta-frequency ranges. Comparative analysis with deep neural network language models highlights distinctive performance patterns, emphasizing the unique capabilities of a rhythmic framework. In essence, our study sheds light on the neural underpinnings of speech processing, emphasizing the role of rhythmic brain mechanisms in structured temporal signal processing -- an insight that challenges prevailing artificial intelligence paradigms and hints at potential advancements in compact and robust computing architectures.","manuscriptTitle":"A brain-rhythm hierarchical predictive computations integrate semantics and acoustics in speech processing","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-06-18 01:50:24","doi":"10.21203/rs.3.rs-4436764/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-computational-science","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"natcomputsci","sideBox":"Learn more about [Nature Computational Science](http://www.nature.com/natcomputsci/)","snPcode":"","submissionUrl":"","title":"Nature Computational Science","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Research","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"2df1c216-58b0-415d-8a40-87a5337ba013","owner":[],"postedDate":"June 18th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":33329407,"name":"Biological sciences/Neuroscience/Computational neuroscience/Dynamical systems"},{"id":33329408,"name":"Biological sciences/Neuroscience/Computational neuroscience/Neural encoding"}],"tags":[],"updatedAt":"2025-09-27T07:07:40+00:00","versionOfRecord":{"articleIdentity":"rs-4436764","link":"https://doi.org/10.1038/s43588-025-00876-9","journal":{"identity":"nature-computational-science","isVorOnly":false,"title":"Nature Computational Science"},"publishedOn":"2025-09-26 04:00:00","publishedOnDateReadable":"September 26th, 2025"},"versionCreatedAt":"2024-06-18 01:50:24","video":"","vorDoi":"10.1038/s43588-025-00876-9","vorDoiUrl":"https://doi.org/10.1038/s43588-025-00876-9","workflowStages":[]},"version":"v1","identity":"rs-4436764","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4436764","identity":"rs-4436764","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}