Prediction of the ENSO using Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) supervised learning from other ocean indices

Prediction of the ENSO using Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) supervised learning from other ocean indices | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Short Report Prediction of the ENSO using Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) supervised learning from other ocean indices Subhadeep Maishal, Biplab Sadhukhan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4210390/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract El Niño-Southern Oscillation (ENSO) is a cyclical global climate phenomenon that frequently results in global climatic anomalies and has significant effects on the economy and society. Thus, forecasting and studying ENSO events is crucial to comprehending and resolving concerns related to global climate change. It is highly significant both practically and scientifically. Numerical modeling techniques and conventional statistical analysis were the primary tools employed in earlier ENSO research. To increase the prediction accuracy of ENSO, this study investigates the application of deep learning. The meteorological and marine time data were processed using recurrent neural networks (RNN) with long- and short-term memory (LSTM). The work takes into account LSTM for predicting multifactor-related ENSO episodes and employs several climate indices as input characteristics. The findings demonstrate the effectiveness of LSTM in predicting ENSO episodes, as well as its potential scientific and practical applications. Over 100 epochs, the LSTM model showed consistent improvement with declining training and validation loss. Its mean squared error (MSE) for training was 0.0954 and 0.0862 for testing, indicating strong generalization. Mean absolute error (MAE) remained stable at 0.2255 for training and 0.2198 for testing, affirming its robustness. Visual analysis revealed close alignment between predicted and actual MEI values, highlighting its ability to capture ENSO dynamics' complexities. Recurrent Neural Networks (RNNs) Long Short-Term Memory (LSTM) supervised learning ocean indices El Niño-Southern Oscillation (ENSO) Figures Figure 1 Figure 2 Figure 3 1. Introduction El Niño-Southern Oscillation (ENSO) is a powerful example of the complex interaction between atmospheric and oceanic processes (Wolter and Timlin 2011 ; Trenberth, 1997 ). It significantly impacts global climate change and has unmatched control over global climate dynamics (Larkin and Harrison, 2002 ). This climatic phenomenon extends over the tropical Pacific Ocean and causes a symphony of temperature anomalies, atmospheric circulation patterns, and hydrological disturbances reverberating throughout continents and oceans (Collins et al., 2010 ). From its origins in the equatorial Pacific to its extensive teleconnections, ENSO captures a range of spatial and temporal scales, representing the uncertainties and the predictabilities of Earth's climate system (Cane, 2005 ; Vecchi and Wittenberg, 2010 ). The mysterious aspect of ENSO is its oscillatory behavior, marked by recurrent El Niño, La Niña, and neutral phases, each of which has unique climatic consequences (Kim et al., 2011 ). El Niño episodes, marked by anomalously warm sea surface temperatures (SSTs) in the central and eastern Pacific, unleash a cascade of atmospheric responses, precipitating droughts, floods, and temperature anomalies worldwide (Varikoden et al., 2015 ; Gizaw and Gan, 2017 ). The La Niña events, characterized by cooler-than-average SSTs in the same region, engender contrasting atmospheric circulation patterns, often heralding opposite climatic extremes (Hoyos et al., 2013; Cai et al., 2015). The genesis and evolution of ENSO are deeply rooted in the complex interplay between the ocean and atmosphere, governed by feedback mechanisms that amplify and attenuate climatic anomalies. ENSO prediction underpins informed decision-making, enhancing resilience to climate variability and mitigating socio-economic (McPhaden et al., 2006). To comprehend and address the issues associated with global climate change, research, and prediction of ENSO events are therefore extremely important from a scientific and practical standpoint (Brown et al., 2008 ). The primary goal of contemporary ENSO event research is to construct indices to examine and investigate the causes and potential future development patterns of ENSO events. Nevertheless, there is no common index standard for diagnosing and predicting ENSO occurrences because different types of ENSO events have distinct physical causes. The quick advancement of artificial intelligence has made it possible to predict ENSO events and choose important indices using machine learning techniques. Additionally, researchers in the field simulate and analyze ENSO events using numerical models; nevertheless, long-term model integration will result in a significant build-up of inaccuracies and ultimately lead to prediction failure. Most numerical prediction models find it challenging to mimic the annual average variations of elements like SST. They cannot predict the appearance and development of ENSO events because of the chaotic and nonlinear properties of the air-sea system. Furthermore, the spatiotemporal evolution of the many ENSO event types is unpredictable, which makes it challenging to anticipate ENSO using standard statistical regression analysis approaches. Many academics have begun using machine learning or deep learning technology to forecast meteorological aspects and short-term climate projections in recent years due to the ongoing development of machine learning methodologies, and they have achieved somewhat good results (Krasnopolsky and Fox-Rabinovitz, 2006 ; Jones, 2017 ; Xiaoqun et al., 2020 ). The ENSO occurrences can be predicted using a long-term, short-term memory neural network (LSTM) if the SST or index prediction problem is characterized as a time series regression problem. A feedforward neural network (FNN) is the foundation for a recurrent neural network (RNN). Though the FNN and RNN are almost similar, both use different methods to transfer information. FNN only uses layers and the links between them to transfer information. The RNN uses a loop structure that transfers the data between the neurons (Li, 2018 ). The RNN can store the input from the previous time point in the network through this connection, which will then influence the network output in the subsequent step. Therefore, the RNN utilizes historical data by mapping a whole piece of information to each output neuron. In contrast, FNN can only map the input to the output layer through the hidden layer. For this reason, regarding prediction problems, including time series data for both the input and the output, RNN performs better than FNN. A unique type of RNN is called LSTM. To prevent information attenuation, it incorporates the functions of long-term and short-term memory as well as the gate idea. The forget gate, input gate, and output gate designs are used to solve it. It performs superbly in speech recognition, machine control, document summarization, language translation, and machine learning. It has also resolved the long-term dependence issue of RNN (Karim et al., 2017 ). The study uses multiple climate indices as input features and considers LSTM for predictive studies on multifactor-related ENSO episodes. 2. Materials and Methods This study explores applying advanced machine learning techniques, specifically RNN and LSTM, to predict the ENSO phenomenon. Leveraging long-term time series data (1979–2022) from the National Oceanic and Atmospheric Administration (NOAA) Physical Sciences Laboratory (PSL), we integrate multiple climate indices as input features. The Antarctic Oscillation (AAO), Arctic Oscillation (AO), Dipole Mode Index (DMI), North Atlantic Oscillation (NAO), Oceanic Niño Index (ONI), Pacific Decadal Oscillation (PDO), Southern Oscillation Index (SOI), Tropical North Atlantic Index (TNA), Tropical South Atlantic Index (TSA), Western Hemisphere Warm Pool (WHWP), and West Pacific (WP) considered for model development (Fig. 1 ) (Cohen 2016 ; Gong and Wang 1999 ; Saji et al. 1999 ). By employing RNN and LSTM architectures, the aim is to enhance the predictive accuracy and robustness of ENSO forecasts (Fig. 2 ). To predict this climate index at each time step t , the input sequence \({X}_{t}\) ​ is supplied into the LSTM network. To deal with univariate time series data, the input size is 1 × 11(climate index input) × 1. The LSTM networks employ three gating mechanisms: the input gate ( \({i}_{t}\) ​), the forget gate ( \({f}_{t}\) ), and the output gate ( \({o}_{t}\) ). These gates regulate the flow of information within the network, allowing it to selectively remember or forget information over time. The hidden state \({h}_{t}\) ​ at time step t is computed using the following equations: $${i}_{t}=\sigma \left({W}_{ix}{x}_{t}+{W}_{ih}{h}_{t-1}+{b}_{i}\right)$$ $${f}_{t}=\sigma \left({W}_{fx}{x}_{t}+{W}_{fh}{h}_{t-1}+{b}_{f}\right)$$ $${o}_{t}=\sigma \left({W}_{ox}{x}_{t}+{W}_{oh}{h}_{t-1}+{b}_{o}\right)$$ $${g}_{t}=\text{t}\text{a}\text{n}\text{h}\left({W}_{gx}{x}_{t}+{W}_{gh}{h}_{t-1}+{b}_{g}\right)$$ $${c}_{t}={f}_{t}\odot {c}_{t-1}+{i}_{t}\odot {g}_{t}$$ $${h}_{t}={o}_{t}\odot \text{t}\text{a}\text{n}\text{h}\left({c}_{t}\right)$$ Where, \({i}_{t}\) , \({f}_{t}\) , \({o}_{t}\) , and \({g}_{t}\) represent the input gate, forget gate, output gate, and cell gate activations, respectively. \({W}_{ix}\) , \({W}_{ih}\) , \({W}_{fx}\) , \({W}_{fh}\) , \({W}_{ox}\) , \({W}_{oh}\) , \({W}_{gx}\) , and \({W}_{gh}\) are weight matrices \({b}_{i}\) , \({b}_{f}\) , \({b}_{o}\) , and \({b}_{g}\) are bias vectors. \({c}_{t}\) is the cell state at time step t . \(\odot\) represents element-wise multiplication. \(\sigma\) is the sigmoid activation function, and \(\text{t}\text{a}\text{n}\text{h}\) is the hyperbolic tangent activation function. Moreover, output prediction \({y}_{t}\) at each time step t is computed by passing the hidden state \({h}_{t}\) ​ through a fully connected layer: $${y}_{t}=\text{s}\text{o}\text{f}\text{t}\text{m}\text{a}\text{x}\left({W}_{yh}{h}_{t}+{b}_{y}\right)$$ Where, \({W}_{yh}\) ​ is the weight matrix connecting the hidden state to the output layer, and \({b}_{y}\) is the bias vector. \(\text{s}\text{o}\text{f}\text{t}\text{m}\text{a}\text{x}\) is the softmax activation function used to convert raw scores into probabilities. The softmax function computes the probability \({y}_{i}\) of each class \(i\) as follows: $${\widehat{y}}_{i}=\frac{{e}^{{z}_{i}}}{\sum _{j=1}^{K} {e}^{{z}_{j}}}$$ Where, \({z}_{i}\) is the raw score (logit) for class \(i\) . \(K\) is the total number of classes. \(e\) is the base of the natural logarithm (Euler's number). The model is trained using backpropagation through time (BPTT) and gradient descent algorithms to minimize a chosen loss function, typically mean squared error (MSE), for regression tasks. The BPTT technique for training recurrent neural networks (RNNs) on different climate indices time series data. It extends the standard backpropagation algorithm to handle sequences by calculating gradients of the loss function concerning the model parameters over a sequence of time steps and updating the parameters accordingly. 3. Results The LSTM model exhibited promising performance in predicting the Multivariate ENSO Index (MEI) using a diverse set of atmospheric and oceanic indices. Training the model over 100 epochs resulted in a gradual decrease in both training and validation loss metrics, indicating effective learning. The mean squared error (MSE) metrics for training and testing data were 0.0954 and 0.0862, respectively, suggesting the model's ability to generalize well to unseen data. Moreover, the mean absolute error (MAE) metrics showed consistent performance between the training (0.2255) and testing (0.2198) datasets, further validating the model's robustness. Visual inspection of predicted MEI values against actual values revealed close alignment, indicating the model's proficiency in capturing the complex temporal dependencies inherent in ENSO dynamics. The observed performance underscores the efficacy of LSTM neural networks in capturing the intricate relationships between atmospheric and oceanic indices and their influence on ENSO variability. These results have significant implications for climate research and operational forecasting. By leveraging advanced machine learning techniques, such as LSTM models, researchers can enhance the accuracy and reliability of ENSO predictions, enabling better-informed decision-making in various sectors, including agriculture, water resource management, and disaster preparedness. Continued refinement and validation of predictive models are essential for improving our understanding of ENSO dynamics and their societal impacts. Future research endeavors may focus on integrating additional data sources, optimizing model architectures, and assessing model robustness under different climatic conditions. The results of this study demonstrate the potential of LSTM neural networks in advancing state-of-the-art ENSO forecasting. By improving our ability to predict and understand ENSO phenomena, we can better mitigate the risks associated with climate variability and change, ultimately contributing to building a more resilient and adaptive society. 4. Discussion The LSTM neural networks in predicting MEI using a comprehensive set of atmospheric and oceanic indices open up avenues for nuanced discussions regarding its implications, limitations, and avenues for future research (Le et al. 2019 ; Srivastava and Anto 2022 ). Firstly, the demonstrated efficacy of LSTM models underscores the potential of advanced machine learning techniques in capturing the complex and nonlinear relationships inherent in ENSO dynamics. By leveraging a diverse array of atmospheric and oceanic indices, the model was able to learn the intricate temporal dependencies that influence ENSO variability. This suggests that LSTM models hold promise for improving the accuracy and reliability of ENSO predictions, which are vital for various sectors such as agriculture, water resource management, and disaster preparedness. However, despite the promising results, it is essential to acknowledge the limitations and challenges associated with LSTM modeling for ENSO prediction. One limitation is the inherent uncertainty and variability in climate data, which can pose challenges for model training and validation. Additionally, LSTM models may struggle with capturing long-term dependencies in time series data, especially in the presence of noisy or sparse input features. Addressing these challenges requires careful preprocessing of data, optimization of model architectures, and rigorous validation against independent datasets. Furthermore, while the current study focused on predicting the MEI using a specific set of atmospheric and oceanic indices, there is potential for expanding the scope of analysis. Future research could explore the integration of additional data sources, such as satellite observations, climate model outputs, and socio-economic indicators, to enhance the model's predictive capabilities. Moreover, investigating the impact of different input features and model hyperparameters on prediction accuracy could provide valuable insights into the underlying mechanisms driving ENSO variability. Another important aspect for discussion is the interpretability of LSTM models in the context of ENSO prediction. While LSTM models offer high predictive accuracy, their complex architectures often make it challenging to interpret the underlying patterns and relationships learned by the model. Enhancing model interpretability through attention mechanisms, feature importance analysis, and model visualization could facilitate a deeper understanding of the physical processes driving ENSO variability. In addition, the deployment and operationalization of LSTM models for real-time ENSO forecasting present practical challenges that must be addressed. Ensuring model reliability, scalability, and computational efficiency are crucial considerations for integrating LSTM-based forecasting systems into decision-support frameworks and operational workflows. The LSTM neural networks for predicting the Multivariate ENSO Index represent a significant advancement in climate prediction science. While the results are promising, there are opportunities for further research to address limitations, enhance model interpretability, and optimize model performance for real-world applications. By continuing to innovate and refine predictive modeling approaches, we can improve our understanding of ENSO dynamics and better mitigate the impacts of climate variability and change on society. Declarations Acknowledgments: We acknowledge the Indian Institute of Technology Kharagpur for facilitating the study. Funding : This study received no funding Author Contributions: SM: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data Curation, Writing - Original Draft, Writing - Review & Editing, Visualization. BS: Conceptualization, Writing - Original Draft, Writing - Review & Editing. Conflicts of interest : The authors confirm no known conflicts of interest associated with this article. Data and materials availability : All the data utilized in this investigation are publicly accessible (https://psl.noaa.gov) Code availability: Example code is available in https://github.com/subhadeep-maishal/VEDAS References Brown, J., Collins, M., Tudhope, A. W., & Toniazzo, T. (2008). Modelling mid-Holocene tropical climate and ENSO variability: towards constraining predictions of future change with palaeo-data. Climate Dynamics, 30, 19-36. Cane, M. A. (2005). The evolution of El Niño, past and future. Earth and Planetary Science Letters, 230(3-4), 227-240. Collins, M., An, S. I., Cai, W., Ganachaud, A., Guilyardi, E., Jin, F. F., ... & Wittenberg, A. (2010). The impact of global warming on the tropical Pacific Ocean and El Niño. Nature Geoscience, 3(6), 391-397. Gizaw, M. S., & Gan, T. Y. (2017). Impact of climate change and El Niño episodes on droughts in sub-Saharan Africa. Climate Dynamics, 49, 665-682. Jones, N. (2017). How machine learning could help to improve climate forecasts. Nature, 548(7668). Karim, F., Majumdar, S., Darabi, H., & Chen, S. (2017). LSTM fully convolutional networks for time series classification. IEEE access, 6, 1662-1669. Kim, H. M., Webster, P. J., & Curry, J. A. (2011). Modulation of North Pacific tropical cyclone activity by three phases of ENSO. Journal of Climate, 24(6), 1839-1849. Krasnopolsky, V. M., & Fox-Rabinovitz, M. S. (2006). Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction. Neural Networks, 19(2), 122-134. Larkin, N. K., & Harrison, D. E. (2002). ENSO warm (El Niño) and cold (La Niña) event life cycles: Ocean surface anomaly patterns, their symmetries, asymmetries, and implications. Journal of climate, 15(10), 1118-1140. Li, H. (2018). Deep learning for natural language processing: advantages and challenges. National Science Review, 5(1), 24-26. Trenberth, K. E. (1997). The definition of el nino. Bulletin of the American Meteorological Society, 78(12), 2771-2778. Varikoden, H., Revadekar, J. V., Choudhary, Y., & Preethi, B. (2015). Droughts of Indian summer monsoon associated with El Niño and Non‐El Niño years. International Journal of Climatology, 35(8), 1916-1925. Vecchi, G. A., & Wittenberg, A. T. (2010). El Niño and our future climate: where do we stand?. Wiley Interdisciplinary Reviews: Climate Change, 1(2), 260-270. Xiaoqun, C., Yanan, G., Bainian, L., Kecheng, P., Guangjie, W., & Mei, G. (2020, March). ENSO prediction based on long short-term memory (LSTM). In IOP Conference Series: Materials Science and Engineering (Vol. 799, No. 1, p. 012035). IOP Publishing. Cohen, Judah. 2016. "An Observational Analysis: Tropical Relative to Arctic Influence on Midlatitude Weather in the Era of Arctic Amplification." Geophysical Research Letters 43(10): 5287–94. https://agupubs.onlinelibrary.wiley.com/doi/10.1002/2016GL069102. Gong, Daoyi, and Shaowu Wang. 1999. "Definition of Antarctic Oscillation Index." Geophysical Research Letters 26(4): 459–62. https://agupubs.onlinelibrary.wiley.com/doi/10.1029/1999GL900003. Le, Xuan Hien, Hung Viet Ho, Giha Lee, and Sungho Jung. 2019. "Application of Long Short-Term Memory (LSTM) Neural Network for Flood Forecasting." Water (Switzerland) 11(7). Saji, N, B Goswami, P Vinayachandran, and T Yamagata. 1999. "A Dipole Mode in the Tropical Ocean." Nature 401(6751): 360–63. Srivastava, Anitej, and S. Anto. 2022. "Weather Prediction Using LSTM Neural Networks." 2022 IEEE 7th International conference for Convergence in Technology, I2CT 2022 : 1–4. Wolter, Klaus, and Michael S. Timlin. 2011. "El Niño/Southern Oscillation Behaviour since 1871 as Diagnosed in an Extended Multivariate ENSO Index (MEI.Ext)." International Journal of Climatology 31(7): 1074–87. https://rmets.onlinelibrary.wiley.com/doi/10.1002/joc.2336. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4210390","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Short Report","associatedPublications":[],"authors":[{"id":290741122,"identity":"a3d186d1-ca7a-402e-bd35-f600d26cf654","order_by":0,"name":"Subhadeep Maishal","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABbElEQVRIie2RP0vDQBiHLwTS5WI2SWhtvsKFQIpQ61dJCDRLKk51UGimyxLsWuiXKLjULeUgXU7dpFCHFCGTiB2UZvBPmkqalqKrYB644+XufXh/8AJQUPAngRwA3Kr0GQdUBJhUenI4FrDfPcn7DwqUvExhf1eWXxDR7CtTNpB7rSBctOvV2j6ZjeJhA6p0fB+GFw9gryQQ5XUIZMFhcbhW0OTEUrzbpnrYbyLCUxNqN94Z0oMoDWYeUKD0fMZFOUW0NZHHxBhMdUQYzELtDjZFnSOpQiQMmAFgsJgPZmvSO/7sDKbWyyjGHah2l8rHWjneUsDE1so89nU0tZGfjIOIdwPRwCvFmGNgbCmIPjfLlVtTue7bp4THYyhSWkLGJYGJoqoMFc0e2QzmtgLpqd2Qa2Xrah7j86rgWdFs8Uaqcnf0KMXD+lHXdaN8sGxcblNaeqeb4Z2keed6NpRSlJVM7OxuLygoKPhPfAGh6HyxuUJz4QAAAABJRU5ErkJggg==","orcid":"","institution":"Indian Institute of Technology Kharagpur","correspondingAuthor":true,"prefix":"","firstName":"Subhadeep","middleName":"","lastName":"Maishal","suffix":""},{"id":290741123,"identity":"ffaa44ed-61b0-46ac-8cf7-6ee3e1e16c36","order_by":1,"name":"Biplab Sadhukhan","email":"","orcid":"","institution":"Indian Institute of Technology Kharagpur","correspondingAuthor":false,"prefix":"","firstName":"Biplab","middleName":"","lastName":"Sadhukhan","suffix":""}],"badges":[],"createdAt":"2024-04-03 06:32:38","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4210390/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4210390/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":54669202,"identity":"24b32aaf-f4b0-4f6e-9a82-85ddd6fcdb13","added_by":"auto","created_at":"2024-04-15 04:25:59","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":246179,"visible":true,"origin":"","legend":"\u003cp\u003eRegions of different oceanic indices in global ocean and red circle represent targeted Multivariate ENSO Index (MEI).\u003c/p\u003e","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4210390/v1/2ba8ac8ea12c16cf52872450.jpeg"},{"id":54669578,"identity":"daab1d3a-268f-477e-803f-3a39572ec290","added_by":"auto","created_at":"2024-04-15 04:33:59","extension":"jpeg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":224043,"visible":true,"origin":"","legend":"\u003cp\u003eArchitecture of\u003cstrong\u003e \u003c/strong\u003eRecurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks.\u003c/p\u003e","description":"","filename":"floatimage2.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4210390/v1/3e2670cefbb3122c1bd84858.jpeg"},{"id":54670030,"identity":"05aad67a-cfe7-420c-b3d7-05eb7c929889","added_by":"auto","created_at":"2024-04-15 04:41:59","extension":"jpeg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":290063,"visible":true,"origin":"","legend":"\u003cp\u003ePrediction of ENSO using\u003cstrong\u003e \u003c/strong\u003eRecurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks.\u003c/p\u003e","description":"","filename":"floatimage3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4210390/v1/3302f48166e735489a4bca90.jpeg"},{"id":56148335,"identity":"136e4414-a45d-48a5-936e-b744cc5e08f1","added_by":"auto","created_at":"2024-05-09 06:34:26","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":591718,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4210390/v1/38992601-3a0d-4db8-a882-3df831d7a687.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Prediction of the ENSO using Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) supervised learning from other ocean indices","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eEl Ni\u0026ntilde;o-Southern Oscillation (ENSO) is a powerful example of the complex interaction between atmospheric and oceanic processes (Wolter and Timlin \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2011\u003c/span\u003e; Trenberth, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e1997\u003c/span\u003e). It significantly impacts global climate change and has unmatched control over global climate dynamics (Larkin and Harrison, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2002\u003c/span\u003e). This climatic phenomenon extends over the tropical Pacific Ocean and causes a symphony of temperature anomalies, atmospheric circulation patterns, and hydrological disturbances reverberating throughout continents and oceans (Collins et al., \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2010\u003c/span\u003e). From its origins in the equatorial Pacific to its extensive teleconnections, ENSO captures a range of spatial and temporal scales, representing the uncertainties and the predictabilities of Earth's climate system (Cane, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2005\u003c/span\u003e; Vecchi and Wittenberg, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e2010\u003c/span\u003e). The mysterious aspect of ENSO is its oscillatory behavior, marked by recurrent El Ni\u0026ntilde;o, La Ni\u0026ntilde;a, and neutral phases, each of which has unique climatic consequences (Kim et al., \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2011\u003c/span\u003e). El Ni\u0026ntilde;o episodes, marked by anomalously warm sea surface temperatures (SSTs) in the central and eastern Pacific, unleash a cascade of atmospheric responses, precipitating droughts, floods, and temperature anomalies worldwide (Varikoden et al., \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2015\u003c/span\u003e; Gizaw and Gan, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). The La Ni\u0026ntilde;a events, characterized by cooler-than-average SSTs in the same region, engender contrasting atmospheric circulation patterns, often heralding opposite climatic extremes (Hoyos et al., 2013; Cai et al., 2015). The genesis and evolution of ENSO are deeply rooted in the complex interplay between the ocean and atmosphere, governed by feedback mechanisms that amplify and attenuate climatic anomalies.\u003c/p\u003e \u003cp\u003eENSO prediction underpins informed decision-making, enhancing resilience to climate variability and mitigating socio-economic (McPhaden et al., 2006). To comprehend and address the issues associated with global climate change, research, and prediction of ENSO events are therefore extremely important from a scientific and practical standpoint (Brown et al., \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2008\u003c/span\u003e). The primary goal of contemporary ENSO event research is to construct indices to examine and investigate the causes and potential future development patterns of ENSO events. Nevertheless, there is no common index standard for diagnosing and predicting ENSO occurrences because different types of ENSO events have distinct physical causes. The quick advancement of artificial intelligence has made it possible to predict ENSO events and choose important indices using machine learning techniques.\u003c/p\u003e \u003cp\u003eAdditionally, researchers in the field simulate and analyze ENSO events using numerical models; nevertheless, long-term model integration will result in a significant build-up of inaccuracies and ultimately lead to prediction failure. Most numerical prediction models find it challenging to mimic the annual average variations of elements like SST. They cannot predict the appearance and development of ENSO events because of the chaotic and nonlinear properties of the air-sea system. Furthermore, the spatiotemporal evolution of the many ENSO event types is unpredictable, which makes it challenging to anticipate ENSO using standard statistical regression analysis approaches. Many academics have begun using machine learning or deep learning technology to forecast meteorological aspects and short-term climate projections in recent years due to the ongoing development of machine learning methodologies, and they have achieved somewhat good results (Krasnopolsky and Fox-Rabinovitz, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2006\u003c/span\u003e; Jones, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2017\u003c/span\u003e; Xiaoqun et al., \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). The ENSO occurrences can be predicted using a long-term, short-term memory neural network (LSTM) if the SST or index prediction problem is characterized as a time series regression problem.\u003c/p\u003e \u003cp\u003eA feedforward neural network (FNN) is the foundation for a recurrent neural network (RNN). Though the FNN and RNN are almost similar, both use different methods to transfer information. FNN only uses layers and the links between them to transfer information. The RNN uses a loop structure that transfers the data between the neurons (Li, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). The RNN can store the input from the previous time point in the network through this connection, which will then influence the network output in the subsequent step. Therefore, the RNN utilizes historical data by mapping a whole piece of information to each output neuron.\u003c/p\u003e \u003cp\u003eIn contrast, FNN can only map the input to the output layer through the hidden layer. For this reason, regarding prediction problems, including time series data for both the input and the output, RNN performs better than FNN. A unique type of RNN is called LSTM. To prevent information attenuation, it incorporates the functions of long-term and short-term memory as well as the gate idea. The forget gate, input gate, and output gate designs are used to solve it. It performs superbly in speech recognition, machine control, document summarization, language translation, and machine learning. It has also resolved the long-term dependence issue of RNN (Karim et al., \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). The study uses multiple climate indices as input features and considers LSTM for predictive studies on multifactor-related ENSO episodes.\u003c/p\u003e"},{"header":"2. Materials and Methods","content":"\u003cp\u003eThis study explores applying advanced machine learning techniques, specifically RNN and LSTM, to predict the ENSO phenomenon. Leveraging long-term time series data (1979\u0026ndash;2022) from the National Oceanic and Atmospheric Administration (NOAA) Physical Sciences Laboratory (PSL), we integrate multiple climate indices as input features. The Antarctic Oscillation (AAO), Arctic Oscillation (AO), Dipole Mode Index (DMI), North Atlantic Oscillation (NAO), Oceanic Ni\u0026ntilde;o Index (ONI), Pacific Decadal Oscillation (PDO), Southern Oscillation Index (SOI), Tropical North Atlantic Index (TNA), Tropical South Atlantic Index (TSA), Western Hemisphere Warm Pool (WHWP), and West Pacific (WP) considered for model development (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e) (Cohen \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2016\u003c/span\u003e; Gong and Wang \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e1999\u003c/span\u003e; Saji et al. \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e1999\u003c/span\u003e). By employing RNN and LSTM architectures, the aim is to enhance the predictive accuracy and robustness of ENSO forecasts (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). To predict this climate index at each time step \u003cem\u003et\u003c/em\u003e, the input sequence \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({X}_{t}\\)\u003c/span\u003e\u003c/span\u003e​ is supplied into the LSTM network. To deal with univariate time series data, the input size is 1 \u0026times; 11(climate index input) \u0026times; 1. The LSTM networks employ three gating mechanisms: the input gate (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({i}_{t}\\)\u003c/span\u003e\u003c/span\u003e​), the forget gate (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({f}_{t}\\)\u003c/span\u003e\u003c/span\u003e), and the output gate (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({o}_{t}\\)\u003c/span\u003e\u003c/span\u003e). These gates regulate the flow of information within the network, allowing it to selectively remember or forget information over time. The hidden state \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({h}_{t}\\)\u003c/span\u003e\u003c/span\u003e​ at time step \u003cem\u003et\u003c/em\u003e is computed using the following equations:\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv id=\"Equa\" class=\"Equation\"\u003e \u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equa\" name=\"EquationSource\"\u003e\n$${i}_{t}=\\sigma \\left({W}_{ix}{x}_{t}+{W}_{ih}{h}_{t-1}+{b}_{i}\\right)$$\u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Equb\" class=\"Equation\"\u003e \u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equb\" name=\"EquationSource\"\u003e\n$${f}_{t}=\\sigma \\left({W}_{fx}{x}_{t}+{W}_{fh}{h}_{t-1}+{b}_{f}\\right)$$\u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Equc\" class=\"Equation\"\u003e \u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equc\" name=\"EquationSource\"\u003e\n$${o}_{t}=\\sigma \\left({W}_{ox}{x}_{t}+{W}_{oh}{h}_{t-1}+{b}_{o}\\right)$$\u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Equd\" class=\"Equation\"\u003e \u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equd\" name=\"EquationSource\"\u003e\n$${g}_{t}=\\text{t}\\text{a}\\text{n}\\text{h}\\left({W}_{gx}{x}_{t}+{W}_{gh}{h}_{t-1}+{b}_{g}\\right)$$\u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Eque\" class=\"Equation\"\u003e \u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Eque\" name=\"EquationSource\"\u003e\n$${c}_{t}={f}_{t}\\odot {c}_{t-1}+{i}_{t}\\odot {g}_{t}$$\u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Equf\" class=\"Equation\"\u003e \u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equf\" name=\"EquationSource\"\u003e\n$${h}_{t}={o}_{t}\\odot \\text{t}\\text{a}\\text{n}\\text{h}\\left({c}_{t}\\right)$$\u003c/div\u003e \u003c/div\u003e \u003c/p\u003e \u003cp\u003eWhere,\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({i}_{t}\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({f}_{t}\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({o}_{t}\\)\u003c/span\u003e\u003c/span\u003e, and\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({g}_{t}\\)\u003c/span\u003e\u003c/span\u003e represent the input gate, forget gate, output gate, and cell gate activations, respectively.\u003c/p\u003e \u003cp\u003e \u003cspan class=\"InlineEquation\"\u003e \u003cspan class=\"mathinline\"\u003e\\({W}_{ix}\\)\u003c/span\u003e \u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({W}_{ih}\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({W}_{fx}\\)\u003c/span\u003e\u003c/span\u003e,\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({W}_{fh}\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({W}_{ox}\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({W}_{oh}\\)\u003c/span\u003e\u003c/span\u003e,\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({W}_{gx}\\)\u003c/span\u003e\u003c/span\u003e, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({W}_{gh}\\)\u003c/span\u003e\u003c/span\u003e are weight matrices \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({b}_{i}\\)\u003c/span\u003e\u003c/span\u003e, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({b}_{f}\\)\u003c/span\u003e\u003c/span\u003e,\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({b}_{o}\\)\u003c/span\u003e\u003c/span\u003e, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({b}_{g}\\)\u003c/span\u003e\u003c/span\u003e are bias vectors. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({c}_{t}\\)\u003c/span\u003e\u003c/span\u003e is the cell state at time step \u003cem\u003et\u003c/em\u003e. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\odot\\)\u003c/span\u003e\u003c/span\u003e represents element-wise multiplication. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\sigma\\)\u003c/span\u003e\u003c/span\u003e is the sigmoid activation function, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\text{t}\\text{a}\\text{n}\\text{h}\\)\u003c/span\u003e\u003c/span\u003e is the hyperbolic tangent activation function. Moreover, output prediction \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({y}_{t}\\)\u003c/span\u003e\u003c/span\u003e at each time step \u003cem\u003et\u003c/em\u003e is computed by passing the hidden state \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({h}_{t}\\)\u003c/span\u003e\u003c/span\u003e​ through a fully connected layer:\u003cdiv id=\"Equg\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equg\" name=\"EquationSource\"\u003e\n$${y}_{t}=\\text{s}\\text{o}\\text{f}\\text{t}\\text{m}\\text{a}\\text{x}\\left({W}_{yh}{h}_{t}+{b}_{y}\\right)$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({W}_{yh}\\)\u003c/span\u003e\u003c/span\u003e​ is the weight matrix connecting the hidden state to the output layer, and \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({b}_{y}\\)\u003c/span\u003e\u003c/span\u003e is the bias vector. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\text{s}\\text{o}\\text{f}\\text{t}\\text{m}\\text{a}\\text{x}\\)\u003c/span\u003e\u003c/span\u003eis the softmax activation function used to convert raw scores into probabilities. The softmax function computes the probability \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({y}_{i}\\)\u003c/span\u003e\u003c/span\u003e of each class \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(i\\)\u003c/span\u003e\u003c/span\u003e as follows:\u003cdiv id=\"Equh\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equh\" name=\"EquationSource\"\u003e\n$${\\widehat{y}}_{i}=\\frac{{e}^{{z}_{i}}}{\\sum _{j=1}^{K} {e}^{{z}_{j}}}$$\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eWhere, \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\({z}_{i}\\)\u003c/span\u003e\u003c/span\u003e is the raw score (logit) for class \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(i\\)\u003c/span\u003e\u003c/span\u003e. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(K\\)\u003c/span\u003e\u003c/span\u003e is the total number of classes. \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(e\\)\u003c/span\u003e\u003c/span\u003e is the base of the natural logarithm (Euler's number).\u003c/p\u003e \u003cp\u003eThe model is trained using backpropagation through time (BPTT) and gradient descent algorithms to minimize a chosen loss function, typically mean squared error (MSE), for regression tasks. The BPTT technique for training recurrent neural networks (RNNs) on different climate indices time series data. It extends the standard backpropagation algorithm to handle sequences by calculating gradients of the loss function concerning the model parameters over a sequence of time steps and updating the parameters accordingly.\u003c/p\u003e"},{"header":"3. Results","content":"\u003cp\u003eThe LSTM model exhibited promising performance in predicting the Multivariate ENSO Index (MEI) using a diverse set of atmospheric and oceanic indices. Training the model over 100 epochs resulted in a gradual decrease in both training and validation loss metrics, indicating effective learning. The mean squared error (MSE) metrics for training and testing data were 0.0954 and 0.0862, respectively, suggesting the model's ability to generalize well to unseen data. Moreover, the mean absolute error (MAE) metrics showed consistent performance between the training (0.2255) and testing (0.2198) datasets, further validating the model's robustness. Visual inspection of predicted MEI values against actual values revealed close alignment, indicating the model's proficiency in capturing the complex temporal dependencies inherent in ENSO dynamics. The observed performance underscores the efficacy of LSTM neural networks in capturing the intricate relationships between atmospheric and oceanic indices and their influence on ENSO variability. These results have significant implications for climate research and operational forecasting. By leveraging advanced machine learning techniques, such as LSTM models, researchers can enhance the accuracy and reliability of ENSO predictions, enabling better-informed decision-making in various sectors, including agriculture, water resource management, and disaster preparedness.\u003c/p\u003e \u003cp\u003eContinued refinement and validation of predictive models are essential for improving our understanding of ENSO dynamics and their societal impacts. Future research endeavors may focus on integrating additional data sources, optimizing model architectures, and assessing model robustness under different climatic conditions. The results of this study demonstrate the potential of LSTM neural networks in advancing state-of-the-art ENSO forecasting. By improving our ability to predict and understand ENSO phenomena, we can better mitigate the risks associated with climate variability and change, ultimately contributing to building a more resilient and adaptive society.\u003c/p\u003e"},{"header":"4. Discussion","content":"\u003cp\u003eThe LSTM neural networks in predicting MEI using a comprehensive set of atmospheric and oceanic indices open up avenues for nuanced discussions regarding its implications, limitations, and avenues for future research (Le et al. \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e2019\u003c/span\u003e; Srivastava and Anto \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Firstly, the demonstrated efficacy of LSTM models underscores the potential of advanced machine learning techniques in capturing the complex and nonlinear relationships inherent in ENSO dynamics. By leveraging a diverse array of atmospheric and oceanic indices, the model was able to learn the intricate temporal dependencies that influence ENSO variability. This suggests that LSTM models hold promise for improving the accuracy and reliability of ENSO predictions, which are vital for various sectors such as agriculture, water resource management, and disaster preparedness. However, despite the promising results, it is essential to acknowledge the limitations and challenges associated with LSTM modeling for ENSO prediction. One limitation is the inherent uncertainty and variability in climate data, which can pose challenges for model training and validation.\u003c/p\u003e \u003cp\u003eAdditionally, LSTM models may struggle with capturing long-term dependencies in time series data, especially in the presence of noisy or sparse input features. Addressing these challenges requires careful preprocessing of data, optimization of model architectures, and rigorous validation against independent datasets. Furthermore, while the current study focused on predicting the MEI using a specific set of atmospheric and oceanic indices, there is potential for expanding the scope of analysis. Future research could explore the integration of additional data sources, such as satellite observations, climate model outputs, and socio-economic indicators, to enhance the model's predictive capabilities. Moreover, investigating the impact of different input features and model hyperparameters on prediction accuracy could provide valuable insights into the underlying mechanisms driving ENSO variability. Another important aspect for discussion is the interpretability of LSTM models in the context of ENSO prediction. While LSTM models offer high predictive accuracy, their complex architectures often make it challenging to interpret the underlying patterns and relationships learned by the model. Enhancing model interpretability through attention mechanisms, feature importance analysis, and model visualization could facilitate a deeper understanding of the physical processes driving ENSO variability. In addition, the deployment and operationalization of LSTM models for real-time ENSO forecasting present practical challenges that must be addressed. Ensuring model reliability, scalability, and computational efficiency are crucial considerations for integrating LSTM-based forecasting systems into decision-support frameworks and operational workflows. The LSTM neural networks for predicting the Multivariate ENSO Index represent a significant advancement in climate prediction science. While the results are promising, there are opportunities for further research to address limitations, enhance model interpretability, and optimize model performance for real-world applications. By continuing to innovate and refine predictive modeling approaches, we can improve our understanding of ENSO dynamics and better mitigate the impacts of climate variability and change on society.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe acknowledge the Indian Institute of Technology Kharagpur for facilitating the study.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e: This study received no funding\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor Contributions: SM:\u003c/strong\u003e Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data Curation, Writing - Original Draft, Writing - Review \u0026amp; Editing, Visualization. \u003cstrong\u003eBS:\u003c/strong\u003e Conceptualization, Writing - Original Draft, Writing - Review \u0026amp; Editing.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflicts of interest\u003c/strong\u003e: The authors confirm no known conflicts of interest associated with this article.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData and materials availability\u003c/strong\u003e: All the data utilized in this investigation are publicly accessible (https://psl.noaa.gov)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCode availability:\u0026nbsp;\u003c/strong\u003eExample code is available in https://github.com/subhadeep-maishal/VEDAS\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eBrown, J., Collins, M., Tudhope, A. W., \u0026amp; Toniazzo, T. (2008). Modelling mid-Holocene tropical climate and ENSO variability: towards constraining predictions of future change with palaeo-data. Climate Dynamics, 30, 19-36.\u003c/li\u003e\n\u003cli\u003eCane, M. A. (2005). The evolution of El Ni\u0026ntilde;o, past and future. Earth and Planetary Science Letters, 230(3-4), 227-240.\u003c/li\u003e\n\u003cli\u003eCollins, M., An, S. I., Cai, W., Ganachaud, A., Guilyardi, E., Jin, F. F., ... \u0026amp; Wittenberg, A. (2010). The impact of global warming on the tropical Pacific Ocean and El Ni\u0026ntilde;o. Nature Geoscience, 3(6), 391-397.\u003c/li\u003e\n\u003cli\u003eGizaw, M. S., \u0026amp; Gan, T. Y. (2017). Impact of climate change and El Ni\u0026ntilde;o episodes on droughts in sub-Saharan Africa. Climate Dynamics, 49, 665-682.\u003c/li\u003e\n\u003cli\u003eJones, N. (2017). How machine learning could help to improve climate forecasts. Nature, 548(7668).\u003c/li\u003e\n\u003cli\u003eKarim, F., Majumdar, S., Darabi, H., \u0026amp; Chen, S. (2017). LSTM fully convolutional networks for time series classification. IEEE access, 6, 1662-1669.\u003c/li\u003e\n\u003cli\u003eKim, H. M., Webster, P. J., \u0026amp; Curry, J. A. (2011). Modulation of North Pacific tropical cyclone activity by three phases of ENSO. Journal of Climate, 24(6), 1839-1849.\u003c/li\u003e\n\u003cli\u003eKrasnopolsky, V. M., \u0026amp; Fox-Rabinovitz, M. S. (2006). Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction. Neural Networks, 19(2), 122-134.\u003c/li\u003e\n\u003cli\u003eLarkin, N. K., \u0026amp; Harrison, D. E. (2002). ENSO warm (El Ni\u0026ntilde;o) and cold (La Ni\u0026ntilde;a) event life cycles: Ocean surface anomaly patterns, their symmetries, asymmetries, and implications. Journal of climate, 15(10), 1118-1140.\u003c/li\u003e\n\u003cli\u003eLi, H. (2018). Deep learning for natural language processing: advantages and challenges. National Science Review, 5(1), 24-26.\u003c/li\u003e\n\u003cli\u003eTrenberth, K. E. (1997). The definition of el nino. Bulletin of the American Meteorological Society, 78(12), 2771-2778.\u003c/li\u003e\n\u003cli\u003eVarikoden, H., Revadekar, J. V., Choudhary, Y., \u0026amp; Preethi, B. (2015). Droughts of Indian summer monsoon associated with El Ni\u0026ntilde;o and Non‐El Ni\u0026ntilde;o years. International Journal of Climatology, 35(8), 1916-1925.\u003c/li\u003e\n\u003cli\u003eVecchi, G. A., \u0026amp; Wittenberg, A. T. (2010). El Ni\u0026ntilde;o and our future climate: where do we stand?. Wiley Interdisciplinary Reviews: Climate Change, 1(2), 260-270.\u003c/li\u003e\n\u003cli\u003eXiaoqun, C., Yanan, G., Bainian, L., Kecheng, P., Guangjie, W., \u0026amp; Mei, G. (2020, March). ENSO prediction based on long short-term memory (LSTM). In IOP Conference Series: Materials Science and Engineering (Vol. 799, No. 1, p. 012035). IOP Publishing.\u003c/li\u003e\n\u003cli\u003eCohen, Judah. 2016. \u0026quot;An Observational Analysis: Tropical Relative to Arctic Influence on Midlatitude Weather in the Era of Arctic Amplification.\u0026quot; \u003cem\u003eGeophysical Research Letters\u003c/em\u003e 43(10): 5287\u0026ndash;94. https://agupubs.onlinelibrary.wiley.com/doi/10.1002/2016GL069102.\u003c/li\u003e\n\u003cli\u003eGong, Daoyi, and Shaowu Wang. 1999. \u0026quot;Definition of Antarctic Oscillation Index.\u0026quot; \u003cem\u003eGeophysical Research Letters\u003c/em\u003e 26(4): 459\u0026ndash;62. https://agupubs.onlinelibrary.wiley.com/doi/10.1029/1999GL900003.\u003c/li\u003e\n\u003cli\u003eLe, Xuan Hien, Hung Viet Ho, Giha Lee, and Sungho Jung. 2019. \u0026quot;Application of Long Short-Term Memory (LSTM) Neural Network for Flood Forecasting.\u0026quot; \u003cem\u003eWater (Switzerland)\u003c/em\u003e 11(7).\u003c/li\u003e\n\u003cli\u003eSaji, N, B Goswami, P Vinayachandran, and T Yamagata. 1999. \u0026quot;A Dipole Mode in the Tropical Ocean.\u0026quot; \u003cem\u003eNature\u003c/em\u003e 401(6751): 360\u0026ndash;63.\u003c/li\u003e\n\u003cli\u003eSrivastava, Anitej, and S. Anto. 2022. \u0026quot;Weather Prediction Using LSTM Neural Networks.\u0026quot; \u003cem\u003e2022 IEEE 7th International conference for Convergence in Technology, I2CT 2022\u003c/em\u003e: 1\u0026ndash;4.\u003c/li\u003e\n\u003cli\u003eWolter, Klaus, and Michael S. Timlin. 2011. \u0026quot;El Ni\u0026ntilde;o/Southern Oscillation Behaviour since 1871 as Diagnosed in an Extended Multivariate ENSO Index (MEI.Ext).\u0026quot; \u003cem\u003eInternational Journal of Climatology\u003c/em\u003e 31(7): 1074\u0026ndash;87. https://rmets.onlinelibrary.wiley.com/doi/10.1002/joc.2336.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM), supervised learning, ocean indices, El Niño-Southern Oscillation (ENSO)","lastPublishedDoi":"10.21203/rs.3.rs-4210390/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4210390/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eEl Ni\u0026ntilde;o-Southern Oscillation (ENSO) is a cyclical global climate phenomenon that frequently results in global climatic anomalies and has significant effects on the economy and society. Thus, forecasting and studying ENSO events is crucial to comprehending and resolving concerns related to global climate change. It is highly significant both practically and scientifically. Numerical modeling techniques and conventional statistical analysis were the primary tools employed in earlier ENSO research. To increase the prediction accuracy of ENSO, this study investigates the application of deep learning. The meteorological and marine time data were processed using recurrent neural networks (RNN) with long- and short-term memory (LSTM). The work takes into account LSTM for predicting multifactor-related ENSO episodes and employs several climate indices as input characteristics. The findings demonstrate the effectiveness of LSTM in predicting ENSO episodes, as well as its potential scientific and practical applications. Over 100 epochs, the LSTM model showed consistent improvement with declining training and validation loss. Its mean squared error (MSE) for training was 0.0954 and 0.0862 for testing, indicating strong generalization. Mean absolute error (MAE) remained stable at 0.2255 for training and 0.2198 for testing, affirming its robustness. Visual analysis revealed close alignment between predicted and actual MEI values, highlighting its ability to capture ENSO dynamics' complexities.\u003c/p\u003e","manuscriptTitle":"Prediction of the ENSO using Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) supervised learning from other ocean indices","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-04-15 04:25:55","doi":"10.21203/rs.3.rs-4210390/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"8e024e7f-2db3-4754-b0d1-812d2651d2ec","owner":[],"postedDate":"April 15th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-05-09T06:26:15+00:00","versionOfRecord":[],"versionCreatedAt":"2024-04-15 04:25:55","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4210390","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4210390","identity":"rs-4210390","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}