Temporal-Spectral Hamiltonian Mixers for Efficient Long Sequence Modeling

Temporal-Spectral Hamiltonian Mixers for Efficient Long Sequence Modeling | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 19 February 2026 V1 Latest version Share on Temporal-Spectral Hamiltonian Mixers for Efficient Long Sequence Modeling Authors : Dingam Camille 0009-0000-0969-1117 [email protected] , Milembolo Miantezila , Makoyel Yokara , and Marie Ange Authors Info & Affiliations https://doi.org/10.22541/au.177153534.48531776/v1 80 views 41 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract We present Temporal Spectral Hamiltonian Mixer, a lightweight and memory-efficient architecture for modeling long-range dependencies in sequential data. Inspired by Hamiltonian dynamics, TSHM introduces a structured module that maintains stability and long-term coherence while remaining simple to implement, efficient to train, and can handle ten to over hundred thousand of sequence length without memory overflow. The design supports both parallelized training and constanttime streaming inference O (1), making it suitable for real-time and low-latency applications. Compared with Transformers and structured state-space models(SSM) such as S4, TSHM achieves a favorable balance between expressivity, computational cost, and engineering simplicity and are naturally bidirectional or sequential in all direction. It avoids specialized kernels, requires only dense linear and pointwise operations, and operates with minimal memory overhead. We provide a mathematical formulation, interpret the model through a Hamiltonian lens, analyze its computational complexity, and outline a reproducible experimental plan across diverse benchmarks in speech recognition, text classification, time-series forecasting, and sequential image classification. Across benchmarks-Long Range Area(LRA), sMNIST, CIFAR (1-D), long horizon forecast dataset and Google Speech Commands, TSHM consistently outperforms Transformer variants, it is second or sometime competitive with SSM (S4), and shows largest gains on time-series forecasting. Notably, TSHM processes raw 16,000-sample speech sequences achieving 91% accuracy, LRA Path-X (16000 sequence length) achieving 80.31% while standard Transformer and Recurrent Neural Network (RNN) baselines fail under the same conditions because of memory overflow or efficiency. Finally, to demonstrate TSHM's real-world practicality, we deployed it as firmware on an ESP32-S3 voice-control device. The model runs at 1.1-1.4 ms per frame and completes 10 head-only Stochastic Gradient Descent (SGD) updates in 1.2 ms, enabling accurate, privacy-preserving on-device learning without cloud dependency. Supplementary Material File (paperv2.pdf) Download 1.63 MB Information & Authors Information Version history V1 Version 1 19 February 2026 Copyright This work is licensed under a Creative Commons Attribution 4.0 International License Keywords and linear complexity long range modeling longterm time series forecasting (ltsf) natural language processing/large language model state space models (s3/s4) Authors Affiliations Dingam Camille 0009-0000-0969-1117 [email protected] Taiyuan University of Technology View all articles by this author Milembolo Miantezila Haute Ecole de Commerce de Kinshasa View all articles by this author Makoyel Yokara View all articles by this author Marie Ange Institut Africain d'Informatique View all articles by this author Metrics & Citations Metrics Article Usage 80 views 41 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Dingam Camille, Milembolo Miantezila, Makoyel Yokara, et al. Temporal-Spectral Hamiltonian Mixers for Efficient Long Sequence Modeling. Authorea . 19 February 2026. DOI: https://doi.org/10.22541/au.177153534.48531776/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.177153534.48531776/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9fe1e9584bac58f4',t:'MTc3OTE4MTIyOA=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();