The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance

The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance Adji Bousso Dieng, Mohammad Reza Rezaei This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6233867/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Current state-of-the-art dynamical models, such as Mamba, assume the same level of noisiness for all elements of a given sequence, which limits their performance on noisy temporal data. In this paper, we introduce the Alpha-Alternator, a novel generative model for time-dependent data that dynamically adapts to the complexity introduced by varying noise levels in sequences. The Alpha-Alternator leverages the Vendi Score (VS), a flexible similarity-based diversity metric, to adjust, at each time step t, the influence of the sequence element at time t and the latent representation of the dynamics up to that time step on the predicted future dynamics. This influence is captured by a parameter that is learned and shared across all sequences in a given dataset. The sign of this parameter determines the direction of influence. A negative value indicates a noisy dataset, where a sequence element that increases the VS is considered noisy, and the model relies more on the latent history when processing that element. Conversely, when the parameter is positive, a sequence element that increases the VS is considered informative, and the Alpha-Alternator relies more on this new input than on the latent history when updating its predicted latent dynamics. The Alpha-Alternator is trained using a combination of observation masking and Alternator loss minimization. Masking simulates varying noise levels in sequences, enabling the model to be more robust to these fluctuations and improving its performance in trajectory prediction, imputation, and forecasting. Our experimental results demonstrate that the Alpha-Alternator outperforms both Alternators and state-of-the-art state-space models across neural decoding and time-series forecasting benchmarks. Physical sciences/Mathematics and computing/Computational science Biological sciences/Neuroscience/Computational neuroscience/Neural decoding Biological sciences/Neuroscience/Computational neuroscience/Dynamical systems Dynamics Vendi Scoring Alternators Neuroscience Time-Series Full Text Additional Declarations There is NO Competing Interest. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6233867","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":434868944,"identity":"bef6a52f-5269-48a0-9793-d9f04cc9bb9d","order_by":0,"name":"Adji Bousso Dieng","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA0UlEQVRIiWNgGAWjYFCChIQDHxgYeBACB4jQcnAGRAtjA7FaGJihVhCphb894eFh2x33ZMzZDx9/8HEHgxzfjQT8WiTOPEg4nHummMeyJy2xceYZBmNJQloYbiQAtbQl8Bjc4DFs5m1jSNxASIs8SIslWAv/R5CWeoJaDEBaGCG2MIK0JBgQ0mII9MvB3jNALWfSDGfObJMwnHnmAX4tcsdzkj/83JFgb3D88IMPH9ts5PmOE7AFGIkJsAgBAQlCykGA/QCyllEwCkbBKBgFmAAAEj1NjVWKaHcAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0001-5687-3554","institution":"Princeton University","correspondingAuthor":true,"prefix":"","firstName":"Adji","middleName":"Bousso","lastName":"Dieng","suffix":""},{"id":434868945,"identity":"ddabcacb-ed95-4f65-a3d8-a85adc72b75d","order_by":1,"name":"Mohammad Reza Rezaei","email":"","orcid":"","institution":"University of Toronto","correspondingAuthor":false,"prefix":"","firstName":"Mohammad","middleName":"Reza","lastName":"Rezaei","suffix":""}],"badges":[],"createdAt":"2025-03-15 16:05:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6233867/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6233867/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":83555915,"identity":"3da4c0d2-c942-466f-84f4-f8ea0b79f1ad","added_by":"auto","created_at":"2025-05-28 11:43:39","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":823661,"visible":true,"origin":"","legend":"Article File","description":"","filename":"AlphaAlternatorNatureCommunicationsSubmission.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6233867/v1_covered_9c879664-23b3-4bc0-8136-593bf3f4d51f.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Dynamics, Vendi Scoring, Alternators, Neuroscience, Time-Series","lastPublishedDoi":"10.21203/rs.3.rs-6233867/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6233867/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Current state-of-the-art dynamical models, such as Mamba, assume the same level of noisiness for all elements of a given sequence, which limits their performance on noisy temporal data. In this paper, we introduce the Alpha-Alternator, a novel generative model for time-dependent data that dynamically adapts to the complexity introduced by varying noise levels in sequences. The Alpha-Alternator leverages the Vendi Score (VS), a flexible similarity-based diversity metric, to adjust, at each time step t, the influence of the sequence element at time t and the latent representation of the dynamics up to that time step on the predicted future dynamics. This influence is captured by a parameter that is learned and shared across all sequences in a given dataset. The sign of this parameter determines the direction of influence. A negative value indicates a noisy dataset, where a sequence element that increases the VS is considered noisy, and the model relies more on the latent history when processing that element. Conversely, when the parameter is positive, a sequence element that increases the VS is considered informative, and the Alpha-Alternator relies more on this new input than on the latent history when updating its predicted latent dynamics. The Alpha-Alternator is trained using a combination of observation masking and Alternator loss minimization. Masking simulates varying noise levels in sequences, enabling the model to be more robust to these fluctuations and improving its performance in trajectory prediction, imputation, and forecasting. Our experimental results demonstrate that the Alpha-Alternator outperforms both Alternators and state-of-the-art state-space models across neural decoding and time-series forecasting benchmarks.","manuscriptTitle":"The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-28 08:44:10","doi":"10.21203/rs.3.rs-6233867/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"e735ee84-6bb2-467c-b834-257c897e1fbf","owner":[],"postedDate":"March 28th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":46305428,"name":"Physical sciences/Mathematics and computing/Computational science"},{"id":46305429,"name":"Biological sciences/Neuroscience/Computational neuroscience/Neural decoding"},{"id":46305430,"name":"Biological sciences/Neuroscience/Computational neuroscience/Dynamical systems"}],"tags":[],"updatedAt":"2025-05-28T11:35:31+00:00","versionOfRecord":[],"versionCreatedAt":"2025-03-28 08:44:10","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6233867","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6233867","identity":"rs-6233867","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}