Fusing Convolutional and Transformer Models with Cross-Attention for Multi-Variate Wind Speed Time Series Forecasting

Fusing Convolutional and Transformer Models with Cross-Attention for Multi-Variate Wind Speed Time Series Forecasting | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 13 June 2025 V1 Latest version Share on Fusing Convolutional and Transformer Models with Cross-Attention for Multi-Variate Wind Speed Time Series Forecasting Authors : Houren Jin , Weihao Qiu , Yuxian Zhao , and Yunxuan Dong 0000-0001-9594-8459 [email protected] Authors Info & Affiliations https://doi.org/10.22541/au.174980723.33605220/v1 256 views 103 downloads Contents Abstract Supplementary Material Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract Accurate multi-variate meteorological time series forecasting, particularly for wind speed, is crucial for effective renewable energy integration. However, existing deep learning models often struggle to simultaneously capture complex long-range temporal dependencies and intricate inter-variable relationships. To address these limitations, this research introduces and evaluates a novel hybrid architecture that combines Spatio-Temporal Convolutional Sequence to Sequence models with Transformer encoders. We investigated both serial and parallel configurations, with the parallel design uniquely employing cross-attention for enhanced feature fusion. Our experiments were conducted on regionally aggregated multi-variate time series data from Southeast Asia, where input spatial dimensions were treated as H=1 and W=1 to focus on temporal and inter-variable dynamics. The parallel Spatio-Temporal Convolutional Sequence to Sequence-Transformer model achieved a Root Mean Squared Error of 0.1064 and a Mean Absolute Error of 0.0858 in wind speed prediction, significantly outperforming various baseline models. These results affirm the substantial benefits of explicitly modeling long-range temporal dependencies and effectively fusing diverse features within multi-variate time series. While this study demonstrates the architecture’s efficacy for regionally focused time series, its design inherently possesses the potential for broader spatio-temporal applications on grid-based data. Supplementary Material File (mypaper.pdf) Download 2.50 MB Information & Authors Information Version history V1 Version 1 13 June 2025 Copyright This work is licensed under a Non Exclusive No Reuse License. Keywords deep learning hybrid models spatio-temporal prediction transformer wind speed forecasting Authors Affiliations Houren Jin Guangxi University School of Computer and Electronic Information View all articles by this author Weihao Qiu Guangxi University School of Computer and Electronic Information View all articles by this author Yuxian Zhao Guangxi University School of Computer and Electronic Information View all articles by this author Yunxuan Dong 0000-0001-9594-8459 [email protected] Guangxi University School of Computer and Electronic Information View all articles by this author Metrics & Citations Metrics Article Usage 256 views 103 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Houren Jin, Weihao Qiu, Yuxian Zhao, et al. Fusing Convolutional and Transformer Models with Cross-Attention for Multi-Variate Wind Speed Time Series Forecasting. Authorea . 13 June 2025. DOI: https://doi.org/10.22541/au.174980723.33605220/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.174980723.33605220/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'a00d0ba1fe6f41e2',t:'MTc3OTYzMzUyMA=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();