Behavior of LSTM and Transformer Deep Learning Models in Flood Simulation Considering South Asian Tropical Climate

Behavior of LSTM and Transformer Deep Learning Models in Flood Simulation Considering South Asian Tropical Climate | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Behavior of LSTM and Transformer Deep Learning Models in Flood Simulation Considering South Asian Tropical Climate G.W.T.I. Madhushanka, M.T.R. Jayasinghe, R.A. Rajapakse This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4115691/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The imperative for a reliable and accurate flood forecasting procedure stem from the hazardous nature of the disaster. In response, researchers are increasingly turning to innovative approaches, particularly machine learning models, which offer enhanced accuracy compared to traditional methods. However, a notable gap exists in the literature concerning studies focused on the South Asian tropical region, which possesses distinct climate characteristics. This study investigates the applicability and behavior of Long Short-Term Memory (LSTM) and Transformer models in flood simulation with one day lead time, at the lower reach of Mahaweli catchment in Sri Lanka, which is mostly affected by the Northeast Monsoon. The importance of different input variables in the prediction was also a key focus of this study. Input features for the models included observed rainfall data collected from three nearby rain gauges, as well as historical discharge data from the target river gauge. Results showed that use of past water level data denotes a higher impact on the output compared to the other input features such as rainfall, for both architectures. All models denoted satisfactory performances in simulating daily water levels, especially low stream flows, with Nash Sutcliffe Efficiency (NSE) values greater than 0.77 while Transformer Encoder model showed a superior performance compared to Encoder Decoder models. Hydrology Artificial Intelligence and Machine Learning Flood Forecasting South Asian Tropical Zone Mahaweli Catchment Long Short-Term Memory Transformer Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 1. Introduction Natural catastrophise such as hurricane, earthquake, and floods lead to significant economic, ecological, and social damages and casualties. Among them, flood can be identified as a phenomenon with severe effects, impacting about 109 million people throughout the world between 1995 and 2015 (Alfieri et al., 2017 ; Hirabayashi et al., 2013 ). As percentages, about 55% of people and 43% of all events were impacted by floods with lost assets totaling over 636 billion USD(Serinaldi et al., 2018 ) encouraging researchers worldwide to mitigate this disaster (Hallegatte et al., 2017 ). When considering South Asian Tropical regions, which are highlighted due to rapid occurrence of seasonal reversal of the wind direction accompanied by intense precipitation, and resultant wet summers and dry winters (Xie & Saiki, 1999 ), large-scale floods and droughts can be expected (Parthasarathy & Mooley, 1978 ). One of viable option in flood hazard management is practical and effective flood warning systems (Boulange et al., 2021 ). Early prediction of floods facilitates timely management of hydro-junction operations and fast evacuation of individuals from flood-affected regions, leading to a reduction in socioeconomic losses (Zhang et al., 2022 ). A significant challenge in advancing flood forecasting technology is the limited availability of field data. Usually, flood prediction approaches can be divided into two categories, physically based models, and data-driven models. Physical models(Mourato et al., 2021 ; Pierini et al., 2014 ) often require substantial amount of both hydrological and geomorphological data for calibration and validation, and they might not always be readily accessible. Furthermore, the model parameters must be carefully tested and evaluated, because they are regionally dependent and can be challenging to estimate. To overcome these limitations, machine learning based data-driven models have gained popularity in flood forecasting because of their ability to capture complex nonlinear patterns, cope with limited data effectively (Rahmati & Pourghasemi, 2017 ), and ability to capture spatial data from images (Lee et al., 1990 ). These models can be effectively implemented solely based on available rainfall data and measured discharge data, without the need for detailed catchment characteristics. Artificial Neural Network (ANN) is a common algorithm for flood simulation because it has outperformed traditional methods on many occasions (Chu et al., 2020 ; Elsafi, 2014 ; Tamiru & Dinka, 2021 ). Then, Recurrent Neural Network (RNN) was introduced for time series forecasting tasks with the ability of capturing essential information from long sequences of data. LSTM, which is a special type of RNN, have gained significant popularity and widespread adoption in hydrologic prediction tasks (Dtissibe et al., 2024 ; Fang et al., 2021 ; Xiang et al., 2020 ; Zou et al., 2023 ). As a solution for some limitations of these traditional neural network algorithms such as low computational speed and ineffectiveness of capturing long-term dependencies, google initiated a new architecture called Transformers(Vaswani et al., 2017 ) which is based on attention mechanism (Bahdanau et al., 2014 ). Although this was originally designed for natural language processing (NLP), the Transformer model has denoted its effectiveness in handling other types of time series data (Farsani & Pazouki, 2021 ; Wu et al., 2020 ). In the realm of flood forecasting, there is a scarcity of studies that incorporate Transformer architecture, representing a notable gap in the literature. Moreover, existing research indicates that the accuracy comparison of models, including Transformers, varies across different datasets (Wei et al., 2023 ; Xu et al., 2023 ). When considering the Tropical regions, especially areas in South Asia, lack of studies regarding deep learning-based flood simulation is an issue. This paper is related to our study (Madhushanka et al., 2024 ) which focused the lower reach of the Mahaweli catchment, Sri Lanka. In this paper, apart from the forecasting capabilities of daily stream flows, the effects of different input features on the output are thoroughly investigated. 2. Literature Review 2.1. South Asian Tropical Climate South Asia, encompassing Afghanistan, Pakistan, India, Nepal, Bhutan, Bangladesh, Maldives, and Sri Lanka, stands as the world's most populous and agriculture-dependent region. The climate in the South Asian region is characterized by the South Asian Monsoon, a significant seasonal phenomenon marked by dramatic shifts in winds that bring vital rainfall to the area. Unlike regions with consistent precipitation year-round, South Asia experiences distinct dry and wet seasons. The summer monsoon, occurring from June to September, carries moisture-laden winds from the Indian, resulting in heavy rainfall. Conversely, the winter monsoon, from December to February, brings dry continental winds from the north (Xie & Saiki, 1999 ). Additionally, there are two inter-monsoon seasons, the first inter-monsoon from March to May and the second inter-monsoon from October to November (Wickramagamage, 2016 ). South Asian countries are particularly susceptible to temperature and precipitation extremes, including floods and droughts, due to the effects of global warming (Naveendrakumar et al., 2019 ). The frequency of intense precipitation events with the potential for extreme outcomes is projected to rise across various regions in South Asian countries (Christensen et al., 2007 ) while Central Asia is expected to have less rainfall compared to the past (Donat et al., 2016 ). Given these factors, there is an urgent need to establish a reliable flood forecasting procedure to prevent future catastrophes. 2.2. Mahaweli Catchment The Mahaweli River, the longest river in Sri Lanka, stretches 335 km in length, originating from the central hills of the country as a collection of numerous small creeks. It traverses through the central region of Sri Lanka before reaching its terminus at the southwestern side of Trincomalee Bay, where it merges with the Bay of Bengal. The Mahaweli River Basin (MRB) is the largest river basin in Sri Lanka, covering an area of approximately 10,448 km 2 , which represents about 16% of the country's total land area (Diyabalanage et al., 2016 ). The runoff from the Mahaweli River contributes to one-seventh of the total runoff of all rivers in Sri Lanka, with an average annual runoff of 8.8 × 10 9 m 3 (De, 1997 ) The distribution of rainfall within the Mahaweli River Basin is uneven both spatially and temporally due to its topographical features. The MRB can be divided into two main parts based on topography: the Upper Mahaweli Basin (UMB) and the Lower Mahaweli Basin (LMB) (Hewawasam, 2010 ). The UMB, situated in the western part of the central highlands, experiences a total annual precipitation of around 6,000 mm (Zubair, 2003 ). Conversely, most parts of the LMB are classified as dry regions, such as the North Central and Eastern provinces, with mean annual precipitation ranging from about 1,600 to 1,900 mm. The precipitation in the UMB is primarily influenced by the southwest monsoon, while the precipitation in the LMB is affected by the northeast monsoon, owing to the intricate terrain and monsoon patterns in Sri Lanka (Shelton & Lin, 2019 ). We selected the LMB as our study area because the climatic data of the region align well with the climatic characteristics of the South Asian Tropical zone. This choice ensures that the case study accurately reflects the conditions typically observed in this region, enhancing the relevance and applicability of our research findings. 2.3. Long Short-Term Memory (LSTM) The Long Short-Term Memory (LSTM) architecture shown in Fig. 1 , was initiated by (Hochreiter & Schmidhuber, 1997 ) as an upgraded version of recurrent neural network (RNN) for addressing the limitation of traditional RNNs in capturing long-term dependencies (Cho et al., 2014 ). Unlike RNNs, LSTM incorporates an additional cell state or cell memory (c t ) where information can be stored along with gates (represented by dashed rectangles in Fig. 1 ). LSTM cells consist of three gates with different functions, namely the forget gate, the update gate, and the output gate. The forget gate, denoted by red rectangle, determines the extent to which elements of the previous cell state vector (c t−1 ) will be forgotten. The update gate (green rectangle) determines the extent to which the information from the current input x t is utilized for updating the cell state (c t ) and then the output gate (blue rectangle) manages the information from the cell state (c t ) which moves into the new hidden state (h t ). 2.4. Transformer and Attention Mechanism The Transformer, denoted in Fig. 2 , is a special type of neural network architecture that was initiated by (Vaswani et al., 2017 ) and it has been a game-changer in the field of natural language processing, particularly for solving machine translation tasks. However, its innovative design has found applications in various other domains, including those that involve the analysis of lengthy input sequences, such as time series forecasting and classification, as highlighted by (S. Li et al., 2019). One of the key features that set the Transformer apart is its self-attention mechanism which replaces traditional recurrent layers in sequence analysis. As the first step of the process, each element in the input sequence is transformed into Query (Q), Key (K), and Value (V) vectors, with a dimension of d model . Then, the Q vectors are matched against the K vectors by performing dot-product multiplications between each pair of Q and K vectors generating a square matrix where each element represents the relationship between the corresponding Q and K vectors. This matrix is then scaled by dividing the square root of the dimension of the Key vectors (d model ) and the scaled matrix is passed through a softmax function to compute the attention scores, which represent the importance or weight of each element in the sequence concerning all other elements. These attention scores are generated for each element in the sequence and the obtained attention scores are multiplied by the corresponding Value (V) vectors to produce a new representation of the input sequence. These separate attention heads are concatenated to generate a large matrix, which is known as Multi-Head Attention (MHA). Finally, the concatenated representations are passed through a final linear transformation. The following Eq. 1 summarizes the self-attention mechanism. $$\text{A}\text{t}\text{t}\text{e}\text{n}\text{t}\text{i}\text{o}\text{n}\left(\text{Q}, \text{K}, \text{V} \right)=\text{s}\text{o}\text{f}\text{t}\text{m}\text{a}\text{x}\left(\frac{\text{Q}{K}^{T}}{\sqrt{{d}_{model}}}\right)V$$ 1 Additionally, positional information of elements in the sequence is achieved through static positional encodings using Sine and Cosine functions, and then they are added to the original input embeddings. The original transformer consists of an encoder decoder architecture as depicted in Fig. 2 . In the decoder, a casual mask is used when calculating the self-attention, preventing each token to attend their future ones. In contrast to self-attention, cross-attention, also known as "encoder-decoder attention," is used to capture relationships between tokens in different input sequences. The output of the encoder is transformed into the query and the key matrices, and the output of the self-attention block of the decoder is transformed into the key matrix in order to calculate the attention as mentioned before. 2.5. Evaluation Metrics The following indicators (Eq. 2 – 5 ) were used to quantify the accuracy of the models. They are Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), Nash-Sutcliffe efficiency (NSE) and Coefficient of Determination (R 2 ). It is preferable for the values of NSE and R 2 to be close to 1 while values of RMSE and MAPE close to 0 are considered desirable. The above indicators were calculated using the following formulas. $$RMSE= \sqrt{\frac{1}{n}\sum _{i=1}^{n}{({X}_{obs}^{i}-{X}_{sim}^{i})}^{2}}$$ 2 $$MAPE= \frac{1}{n}\sum _{i=1}^{n}\left(\frac{|{X}_{sim}^{i}-{X}_{obs}^{i}|}{\left|{X}_{obs}^{i}\right|}\right)$$ 3 $$NSE=1-\frac{\sum _{i=1}^{n}({{X}_{obs}^{i}- {X}_{sim}^{i})}^{2}}{\sum _{i=1}^{n}({{X}_{obs}^{i}- \stackrel{-}{{X}_{obs}} )}^{2}}$$ 4 where n denotes the number of the observations; X i obs and X i sim represent the observation and simulation on day i , while X¯ obs and X¯ obs denote the mean values of the observation and simulation series. 2.6. Lag Correlation In hydrology, the timing misalignment between predicted and observed data is an issue. This temporal discrepancy can be quantified and analyzed using lag correlation metrics, providing insights into the timing errors. The process involves computing a selected error matrix (e.g., root mean square error), then systematically adjusting or lagging one of the time series relative to the other and recomputing the error matrix (Jackson et al., 2019 ). This approach indicates the time lag at which the correlation or similarity is maximized between observations and predictions. Hyndman & Khandakar ( 2008 ) utilized lag correlation measures to gain insights into their dataset's ability to capture specific events, despite the timing variations. In our case, we used RMSE value to determine lag correlation between simulations and observations with 1 day lag time. We calculated the percentage deviation of RMSE between the original and the lagged prediction series in order to quantify the lag correlation. 3. Methodology 3.1. Study Area The Somawathi area, located downstream of the Parakrama Samudra reservoir in the Polonnaruwa district and within the Lower Mahaweli Basin (LMB), is recognized as a flood-prone region. Floods in this area typically occur between December and February, coinciding with the northeast monsoon (NEM) season. To monitor flood levels effectively, the Manampitiya river gauge station plays a crucial role. Real-time water level data, along with alert levels, minor flood levels, and major flood levels for the station, are available on the Irrigation Department of Sri Lanka's website. Figure 3 shows the map of the study area. 3.2. The Dataset The dataset pertaining to the area comprises daily water levels (in meters) recorded at the Manampitiya station and daily rainfall measurements (in millimeters) obtained from three upstream meteorological stations, namely Angamedilla, Aralaganwila and Polonnaruwa Agri. Figure 4 shows the water level and rainfall data while Table 1 exhibits the statistics of them. Table 1 Dataset summary Water Level (m) and Rainfall (mm) Station Name Aralaganwila Angamedilla Polonnaruwa Agri Manampitiya count 10319 10319 10319 10319 mean 4.94584 4.820227 4.20901 33.382384 std 15.083687 15.451919 13.486747 0.643105 min 0 0 0 32.196 max 225.8 222 184 37.254333 Based on the boxplots presented in Fig. 4 , we can discern the influence of the Northeast monsoon on the region. Upon closer examination, it becomes evident that while the maximum precipitation occurs during December to February, there are also instances of extreme rainfall events throughout the year. Although some of these events are classified as outliers, it is not advisable to use preprocessing techniques such as the Interquartile Range (IQR) to remove them, as some of these points might be attributed to unexpected extreme weather conditions. Such occurrences, including sudden storms with heavy rainfall, are common in Sri Lanka due to the influences of the Bay of Bengal. Furthermore, consistent patterns observed in monthly precipitation and water level data indicate a strong correlation between them. This correlation underscores the interdependence of precipitation patterns and water levels in the region, emphasizing the importance of considering both variables in this flood simulation task. As outlined in our previous paper (Madhushanka et al., 2024 ), the analysis was conducted using rainfall data from the designated rain gauge stations along with the water level at Manampitiya, based on the Pearson correlation coefficient calculated among the four stations. Because upstream river gauges belong to a region with different geographic and climatic conditions, they were considered not to be used. Prior to their use as inputs and labels for the models, the data underwent normalization using the Standard Scaling method, which involves considering the mean and standard deviation values of each variable. Subsequently, the dataset was split into the training set and the test set. Specifically, the first 70% of the data was allocated for the training set, while the remaining portion was reserved for the test set to evaluate model performance. 3.3. Experimental Setup The work was conducted using the Python programming language, with data preprocessing, management, and visualization carried out using libraries such as NumPy, Pandas, Scikit-learn, Matplotlib, and Seaborn. For deep learning tasks, the TensorFlow framework was employed. Training of the models was conducted on the Google Colab platform, which provides a cloud-based environment for running Python code, particularly well-suited for machine learning and deep learning tasks. Historical data of the three rain gauges and the target river gauge were used as inputs to forecast the following day water level using the sliding window method. Hyperparameters for the models were kept same as in our previous paper (Madhushanka et al., 2024 ), which were determined by trial-and-error approach. The Transformer architecture employed has been slightly modified from the original implementation presented in (Vaswani et al., 2017 ). Notably, the original input data were used directly without being mapped to an embedding vector, which was done considering the continuity of the data for this regression task. Additionally, the mask of the self-attention layer in the decoder was omitted, thereby allowing time series data to access their successors. However, other aspects such as positional encoding, the number of layers and attention heads, and the dropout rate were kept consistent with the specifications outlined in the original paper. All the hyperparameters are shown in Table 3 . “Early Stopping” was used as the regularization for all the LSTM and Transformer models. As the next task, we studied the contributions from each input features to the final output by considering the following input combinations. Along with the initial 2 models, another 2 LSTMs and 2 Transformer Encoders were utilized for this task with the same model architectures, except the input layer. i. Case 1 - Past water levels and rainfall data as inputs (Initial models) ii. Case 2 - Past water levels as the only input iii. Case 3 - Past rainfall data as the only input Based on the results, Combination 1 was selected as the best scenario for both LSTM and Transformer encoder models. Then we considered two additional Transformer encoder-decoder models with the same hyperparameters as in the encoder (Fig. 5 ), for a broader comparison. Table 3 The values of hyperparameters Hyperparameter Value Batch Size 32 Sliding window size 15 Number of LSTM units in the hidden layer 64 Optimizer Adam Activation function ReLU Validation split 0.1 Learning rate 0.001 For the transformer d model 64 d ff 192 Number of layers 6 Number of heads 8 Dropout rate 0.1 4. Results and Discussion 4.1. Impact of Input Features on simulation of daily water level Three LSTM and Three Transformer models were used to examine the impacts of the 3 different input combinations for prediction. RMSE values of the results were calculated after lagging the prediction series by one day in order to examine the lag correlation between the 2 series as shown in Fig. 7 . According to the results (blue and green bars) in Fig. 7 , Case 3 has the highest error and case 2 denotes the lowest error while the error of Case 1 is close to Case 2 , for both LSTM and Transformer algorithms. For the LSTM, RMSE performance was improved by 47% in Case 3 compared to Case 1 while 12% from Case 1 to Case 2 . For the Transformer, they were 39% and 9% respectively, denoting a similar behavior. This high improvement from Case 3 to Case 1 and comparatively small improvement from Case 1 to Case 2 indicate a higher impact of the past water level data on the output, among all the input features. When considering the LSTM models (blue and orange bars), there is a large reduction of RMSE between the actual and the lagged scenarios in Case 1 and 2 while Case 3 does not show a lag correlation. RMSE of the LSTM was dropped by 76% in Case 1 and 51% in Case 2 , while the Transformer encoder (green and red bars) had the values of 67% and 46% respectively, indicating the highest percentage reduction for the univariate analysis (Case 1 ). These percentage values exhibit the impact of the 1 day lagging of the prediction series. This error reduction might happen when the models allocate a higher weightage to the final timestep of the input series, which was generated by the sliding window method. It explains the univariate analysis showing the highest RMSE reduction while Case 3 , which used only rainfalls as inputs, showing no reduction when lagging the prediction series by one day. We can suggest that low lag correlation is better because one can argue that the prediction series is developed by solely shifting the observation series by a time offset. Case 2 showed the lowest RMSE value as well as a low lag correlation compared to Case 1 . Therefore, we used past rainfall and water level data for the upcoming models based on these results. 4.2. Model performance Figure 8 – 11 denote the daily water level of observation and simulation at Manampitiya station in Mahaweli catchment for the training (1984–2003) and testing (2004–2012) periods in the Multivariate Analysis. As shown in Fig. 8 – 11 , all multivariate models performed relatively well in simulating average water levels of Mahaweli River. However, they differ greatly when it comes to simulating both upward and downward peaks in streamflow. It should be noted that both Transformer Encoder Decoder models significantly underestimated the peak water levels, mostly for daily water levels exceeding 35.5 m, while clear overestimations were observed for low water levels, especially for water levels less than 33 m. Among the models tested, LSTM and Transformer encoder showed superior performance in simulating the peak water levels, while the other 2 models performed poorly in this regard. Table 4 exhibits the evaluation indices of the developed models to compare their performances. Four evaluation metrics (RMSE, MAPE, NSE and R 2 ) were utilized for measuring the forecasting capabilities of the models. During the training period, RMSE ranged from 18.09 to 23.59, MAPE varied from 0.282–0.392%, and R 2 varied from 0.8743 to 0.9185. All models had NSE greater than 0.86. LSTM achieved an NSE of 0.9183 and RMSE of 18.09, with R 2 of 0.9185 and a MAPE of 0.282%. For the Transformer Encoder, those values were 0.9010, 19.90, 0.9014 and 0.314%, respectively. Both Transformer Encoder – Decoder models denoted a poorer performance compared to the others. During the testing period, all models demonstrated satisfactory performance in simulating daily water level, with NSE values greater than 0.7768, although the performances are lower compared to the training phase. The LSTM and Transformer Encoder showed similar performances with RMSE, MAPE, NSE and R 2 values of 27.83, 0.449%, 0.7971, 0.8026 and 27.83, 0.471%, 0.7972, 0.7985 respectively. Transformer Enc - WL Dec – RF and Enc - RF Dec – WL denoted similar evaluation values with a RMSE of 29.10 and 29.19 and NSE of 0.7782 and 0.7768 respectively. Transformer encoder exhibited improved performance compared to Encoder-Decoder models. Table 4 The evaluation of daily streamflow simulation Period Model RMSE (cm) MAPE (%) NSE R 2 Training LSTM 18.09 0.282 0.9183 0.9185 Transformer Encoder 19.90 0.314 0.9010 0.9014 Transformer Enc -> WL Dec -> RF 21.51 0.338 0.8844 0.8928 Transformer Enc -> RF Dec -> WL 23.59 0.392 0.8609 0.8743 Testing LSTM 27.83 0.449 0.7971 0.8026 Transformer Encoder 27.83 0.471 0.7972 0.7985 Transformer Enc -> WL Dec -> RF 29.10 0.527 0.7782 0.7828 Transformer Enc -> RF Dec -> WL 29.19 0.52 0.7768 0.7901 Overall, LSTM and Transformer encoder models exhibited approximately similar performances in daily water level forecasting. Single Encoder model showed the better performance among all the transformer models. When there is an encoder as well as a decoder in a Transformer model, it takes the Query (Q) and the Key (K) vectors from the encoder and the Value (V) vector from the decoder to calculate the cross attention between the two inputs. But when there is just the encoder, all the Q, K and V are taken from a single input series. That might be the reason for its increased accuracy. However, if there are input series with 2 different sequence lengths, an Encoder-Decoder model has to be used for the analysis. 5. Conclusions and Recommendations In this study, four machine learning models (LSTM, Transformer Encoder and Transformer Encoder-Decoder Model 1 and 2) were applied to simulate daily water levels and extremities at Manampitiya river gauge in Mahaweli catchment. The impacts of different inputs on the output were also of interest. The main conclusions of the study are as follows. Use of past water level data denotes a higher impact on the output compared to the other input features such as rainfall. It is recommended to use many other input features that show higher correlation values with the target, along with the past observations of the output since this strategy increases the model performance as well as decreases the lag correlation. LSTM and Transformer encoder shows similar accuracies in daily water level forecasting and predicting extremities. Although Transformer models tend to show a better performance in streamflow forecasting as in studies of (Castangia et al., 2023 ; Liu et al., 2022 ; Xu et al., 2023 ), lack of sufficient data may cause the performance reduction for the Transformer Encoder model (Wei et al., 2023 ). In this case, switching the inputs between the Encoder and Decoder shows similar performance but it might be different in another case. Therefore, it is recommended to do this strategy and check the accuracy when there are input features with 2 different time steps. Declarations Author Contribution G.W.T.I. Madhushanka led the development of research methodology, raw data acquisition, code development, data manipulation, training and evaluation of the models, results analysis and writing the original draft. M.T.R. Jayasinghe led the supervision throughout the study, providing financial support and facilitating data acquisition by signing the agreements. R.A. Rajapakse contributed to data analysis, code development, final manuscript writing and provided guidance as well as financial support. Funding Sources This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgements We would like to thank Ranga Rodrigo for giving valuable advice as well as learning resources regarding machine learning. References Alfieri, L., Bisselink, B., Dottori, F., Naumann, G., de Roo, A., Salamon, P., Wyser, K., & Feyen, L. (2017). Global projections of river flood risk in a warmer world. Earth’s Future , 5 (2), 171–182. https://doi.org/10.1002/2016EF000485 Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate . Boulange, J., Hanasaki, N., Yamazaki, D., & Pokhrel, Y. (2021). Role of dams in reducing global flood exposure under climate change. Nature Communications , 12 (1), 417. https://doi.org/10.1038/s41467-020-20704-0 Castangia, M., Grajales, L. M. M., Aliberti, A., Rossi, C., Macii, A., Macii, E., & Patti, E. (2023). Transformer neural networks for interpretable flood forecasting. Environmental Modelling and Software , 160 . https://doi.org/10.1016/j.envsoft.2022.105581 Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation . "Christensen, J. H., "Hewitson, B., "Busuioc, A., "Chen, A., "Gao, X., "Held, I., "Jones, R., "Kolli, R. K., "Kwon, W. T., "Laprise, R., "Magana Rueda, V., "Mearns, L., "Menendez, C. G., "Raisanen, J., "Rinke, A., "Sarr, A., & "Whetton, P. (2007). Regional Climate Projections. Chapter 11 . Cambridge University Press, Cambridge (United Kingdom). https://doi.org/https://doi.org/ Chu, H., Wu, W., Wang, Q. J., Nathan, R., & Wei, J. (2020). An ANN-based emulation modelling framework for flood inundation modelling: Application, challenges and future directions. Environmental Modelling & Software , 124 , 104587. https://doi.org/10.1016/J.ENVSOFT.2019.104587 De, A. C. (1997). Management of the mahaweli, a river in sri lanka. Water International , 22 (2), 98–107. https://doi.org/10.1080/02508069708686678 Diyabalanage, S., Abekoon, S., Watanabe, I., Watai, C., Ono, Y., Wijesekara, S., Guruge, K. S., & Chandrajith, R. (2016). Has irrigated water from Mahaweli River contributed to the kidney disease of uncertain etiology in the dry zone of Sri Lanka? Environmental Geochemistry and Health , 38 (3), 679–690. https://doi.org/10.1007/s10653-015-9749-1 Donat, M. G., Lowry, A. L., Alexander, L. V., O’Gorman, P. A., & Maher, N. (2016). More extreme precipitation in the world’s dry and wet regions. Nature Climate Change , 6 (5), 508–513. https://doi.org/10.1038/nclimate2941 Dtissibe, F. Y., Ari, A. A. A., Abboubakar, H., Njoya, A. N., Mohamadou, A., & Thiare, O. (2024). A comparative study of Machine Learning and Deep Learning methods for flood forecasting in the Far-North region, Cameroon. Scientific African , 23 , e02053. https://doi.org/10.1016/J.SCIAF.2023.E02053 Elsafi, S. H. (2014). Artificial Neural Networks (ANNs) for flood forecasting at Dongola Station in the River Nile, Sudan. Alexandria Engineering Journal , 53 (3), 655–662. https://doi.org/10.1016/J.AEJ.2014.06.010 Fang, Z., Wang, Y., Peng, L., & Hong, H. (2021). Predicting flood susceptibility using LSTM neural networks. Journal of Hydrology , 594 , 125734. https://doi.org/10.1016/J.JHYDROL.2020.125734 Farsani, R. M., & Pazouki, E. (2021). A Transformer Self-Attention Model for Time Series Forecasting. Journal of Electrical and Computer Engineering Innovations , 9 (1), 1–10. https://doi.org/10.22061/JECEI.2020.7426.391 Hallegatte, S., Vogt-Schilb, A., Bangalore, M., & Rozenberg, J. (2017). Unbreakable: Building the Resilience of the Poor in the Face of Natural Disasters . Washington, DC: World Bank. https://doi.org/10.1596/978-1-4648-1003-9 Hewawasam, T. (2010). Effect of land use in the upper Mahaweli catchment area on erosion, landslides and siltation in hydropower reservoirs of Sri Lanka. Journal of the National Science Foundation of Sri Lanka , 38 (1), 3. https://doi.org/10.4038/jnsfsr.v38i1.1721 Hirabayashi, Y., Mahendran, R., Koirala, S., Konoshima, L., Yamazaki, D., Watanabe, S., Kim, H., & Kanae, S. (2013). Global flood risk under climate change. Nature Climate Change , 3 (9), 816–821. https://doi.org/10.1038/nclimate1911 Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation , 9 (8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 Hyndman, R. J., & Khandakar, Y. (2008). Journal of Statistical Software Automatic Time Series Forecasting: The forecast Package for R (Vol. 27). http://www.jstatsoft.org/ Jackson, E. K., Roberts, W., Nelsen, B., Williams, G. P., Nelson, E. J., & Ames, D. P. (2019). Introductory overview: Error metrics for hydrologic modelling – A review of common practices and an open source library to facilitate use and adoption. In Environmental Modelling and Software (Vol. 119, pp. 32–48). Elsevier Ltd. https://doi.org/10.1016/j.envsoft.2019.05.001 Lee, J., Weger, R. C., Sengupta, S. K., & Welch, R. M. (1990). A neural network approach to cloud classification. IEEE Transactions on Geoscience and Remote Sensing , 28 (5), 846–855. https://doi.org/10.1109/36.58972 Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y.-X., & Yan, X. (n.d.). Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting . Liu, C., Liu, D., & Mu, L. (2022). Improved Transformer Model for Enhanced Monthly Streamflow Predictions of the Yangtze River. IEEE Access , 10 , 58240–58253. https://doi.org/10.1109/ACCESS.2022.3178521 Madhushanka, T., Jayasinghe, T., & Rajapakse, R. (2024). Multi Day Ahead Flood Predictionin South Asian Tropical Zone Using Deep Learning. Available as a Preprint . https://doi.org/https://doi.org/10.21203/rs.3.rs-4070758/v1 Mourato, S., Fernandez, P., Marques, F., Rocha, A., & Pereira, L. (2021). An interactive Web-GIS fluvial flood forecast and alert system in operation in Portugal. International Journal of Disaster Risk Reduction , 58 , 102201. https://doi.org/10.1016/j.ijdrr.2021.102201 Naveendrakumar, G., Vithanage, M., Kwon, H.-H., Chandrasekara, S. S. K., Iqbal, M. C. M., Pathmarajah, S., Fernando, W. C. D. K., & Obeysekera, J. (2019). South Asian perspective on temperature and rainfall extremes: A review. Atmospheric Research , 225 , 110–120. https://doi.org/10.1016/j.atmosres.2019.03.021 Parthasarathy, B., & Mooley, D. A. (1978). Some Features of a Long Homogeneous Series of Indian Summer Monsoon Rainfall. Monthly Weather Review , 106 (6), 771–781. https://doi.org/10.1175/1520-0493(1978)1062.0.CO;2 Pierini, N. A., Vivoni, E. R., Robles‐Morua, A., Scott, R. L., & Nearing, M. A. (2014). Using observations and a distributed hydrologic model to explore runoff thresholds linked with mesquite encroachment in the Sonoran Desert. Water Resources Research , 50 (10), 8191–8215. https://doi.org/10.1002/2014WR015781 Rahmati, O., & Pourghasemi, H. R. (2017). Identification of Critical Flood Prone Areas in Data-Scarce and Ungauged Regions: A Comparison of Three Data Mining Models. Water Resources Management , 31 (5), 1473–1487. https://doi.org/10.1007/s11269-017-1589-6 Serinaldi, F., Loecker, F., Kilsby, C. G., & Bast, H. (2018). Flood propagation and duration in large river basins: a data-driven analysis for reinsurance purposes. Natural Hazards , 94 (1), 71–92. https://doi.org/10.1007/s11069-018-3374-0 Shelton, S., & Lin, Z. (2019). Streamflow variability over the Period of 1990-2014 in Mahaweli River basin, Sri Lanka and its possible mechanisms. Water (Switzerland) , 11 (12). https://doi.org/10.3390/w11122485 Tamiru, H., & Dinka, M. O. (2021). Application of ANN and HEC-RAS model for flood inundation mapping in lower Baro Akobo River Basin, Ethiopia. Journal of Hydrology: Regional Studies , 36 , 100855. https://doi.org/10.1016/J.EJRH.2021.100855 Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention Is All You Need . http://arxiv.org/abs/1706.03762 Wei, X., Wang, G., Schmalz, B., Hagan, D. F. T., & Duan, Z. (2023). Evaluate Transformer model and Self-Attention mechanism in the Yangtze River basin runoff prediction. Journal of Hydrology: Regional Studies , 47 . https://doi.org/10.1016/j.ejrh.2023.101438 Wickramagamage, P. (2016). Spatial and temporal variation of rainfall trends of Sri Lanka. Theoretical and Applied Climatology , 125 (3–4), 427–438. https://doi.org/10.1007/s00704-015-1492-0 Wu, N., Green, B., Ben, X., & O’Banion, S. (2020). Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case . http://arxiv.org/abs/2001.08317 Xiang, Z., Yan, J., & Demir, I. (2020). A Rainfall-Runoff Model With LSTM-Based Sequence-to-Sequence Learning. Water Resources Research , 56 (1). https://doi.org/10.1029/2019WR025326 Xie, S.-P., & Saiki, N. (1999). Abrupt Onset and Slow Seasonal Evolution of Summer Monsoon in an Idealized GCM Simulation. Journal of the Meteorological Society of Japan. Ser. II , 77 (4), 949–968. https://doi.org/10.2151/jmsj1965.77.4_949 Xu, Y., Lin, K., Hu, C., Wang, S., Wu, Q., Zhang, L., & Ran, G. (2023). Deep transfer learning based on transformer for flood forecasting in data-sparse basins. Journal of Hydrology , 625 . https://doi.org/10.1016/j.jhydrol.2023.129956 Zhang, Y., Ragettli, S., Molnar, P., Fink, O., & Peleg, N. (2022). Generalization of an Encoder-Decoder LSTM model for flood prediction in ungauged catchments. Journal of Hydrology , 614 , 128577. https://doi.org/10.1016/j.jhydrol.2022.128577 Zou, Y., Wang, J., Lei, P., & Li, Y. (2023). A novel multi-step ahead forecasting model for flood based on time residual LSTM. Journal of Hydrology , 620 , 129521. https://doi.org/10.1016/J.JHYDROL.2023.129521 Zubair, L. (2003). El Niño-southern oscillation influences on the Mahaweli streamflow in Sri Lanka. International Journal of Climatology , 23 (1), 91–102. https://doi.org/10.1002/joc.865 Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4115691","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":280445964,"identity":"c2b59fc1-e73f-44e4-9638-b7a25858cec0","order_by":0,"name":"G.W.T.I. Madhushanka","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABOElEQVRIie2PMUvDQBTHLxTsku6VKH6FSuAcGpMP4pIjkFsiFARdOkSEuOjexY8gBAqHbpGDTqe3BnRIEDJ1iLgoZPCldUoT0E3kfvCOu8f78X+HkELxFxlCuahXXzFUgpAOpzuy4KGdJy2G3qZoSTbxayXsVFBD6T1kJV/12xTHuCjK/G5Mbi85e9Wrl70DnWewmLShAylT66iZsrMwZ0RQwoR/Zg6iYv/+OhqB8uwxQUBZ+MdhczHXRCTihCUBNgYh12KJ1gpOQNFCvqnQ97Uil9jQK+7Esl+C8uRhmXcowXdKCin6Fifx41Wdktg47UhJgxNQqMnS4nT7JuJeLMQEFM/FKaS4m3/pz+hc+4zGu0x6bLis+GEs6Pzto7IdLGmelVOrqXRDVpPuT8drnN8MKxQKxb/mC5EJjpVCdhDzAAAAAElFTkSuQmCC","orcid":"","institution":"University of Moratuwa, Sri Lanka","correspondingAuthor":true,"prefix":"","firstName":"G.W.T.I.","middleName":"","lastName":"Madhushanka","suffix":""},{"id":280445965,"identity":"95dad4d5-c8ea-4307-bc8f-b2819d54c30a","order_by":1,"name":"M.T.R. Jayasinghe","email":"","orcid":"","institution":"University of Moratuwa, Sri Lanka","correspondingAuthor":false,"prefix":"","firstName":"M.T.R.","middleName":"","lastName":"Jayasinghe","suffix":""},{"id":280445966,"identity":"7c3fbb85-14c0-4582-87f9-b8c1501f5b2a","order_by":2,"name":"R.A. Rajapakse","email":"","orcid":"","institution":"RASU Consulting, New York, USA","correspondingAuthor":false,"prefix":"","firstName":"R.A.","middleName":"","lastName":"Rajapakse","suffix":""}],"badges":[],"createdAt":"2024-03-17 06:31:01","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":true,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":true},"doi":"10.21203/rs.3.rs-4115691/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4115691/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":52893088,"identity":"64246562-9b4c-4244-aa49-69a32d00e670","added_by":"auto","created_at":"2024-03-18 12:08:41","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":131939,"visible":true,"origin":"","legend":"\u003cp\u003eLSTM architecture\u003c/p\u003e","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/347f57e8148fb26fa5ae93c6.jpg"},{"id":52893091,"identity":"2ec6f531-7159-4c9c-9260-aa70ffdb3c30","added_by":"auto","created_at":"2024-03-18 12:08:41","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":144732,"visible":true,"origin":"","legend":"\u003cp\u003eTransformer Architecture – with the Encoder and the Decoder\u003c/p\u003e","description":"","filename":"image2.png","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/4184b37bb7f9803caf0e35b7.png"},{"id":52893500,"identity":"9df73af7-2e86-4310-b0a4-0a5c045bedaa","added_by":"auto","created_at":"2024-03-18 12:16:41","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":267242,"visible":true,"origin":"","legend":"\u003cp\u003eMahaweli water shed up to Manampitiya station\u003c/p\u003e","description":"","filename":"image3.png","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/e8eb9eca953b1afc1b70db12.png"},{"id":52893096,"identity":"06add4f7-b0b7-47b5-8918-7021c5e3594b","added_by":"auto","created_at":"2024-03-18 12:08:42","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":279814,"visible":true,"origin":"","legend":"\u003cp\u003eCharacteristics of the rainfall and water level data – boxplots show the monthly and annual variations of the data\u003c/p\u003e","description":"","filename":"image4.png","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/6ffff086edf817b311434ffb.png"},{"id":52893090,"identity":"db28a99d-af5b-4142-894f-5f26d422c452","added_by":"auto","created_at":"2024-03-18 12:08:41","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":259012,"visible":true,"origin":"","legend":"\u003cp\u003eDeveloped Transformer models – In addition to the initial Encoder model, two additional Encoder-Decoder models were developed for the comparison\u003c/p\u003e","description":"","filename":"5.jpg","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/907c18b1c5e82b398c293576.jpg"},{"id":52893093,"identity":"53d3a52f-2e71-4bbd-b969-cefc07c15015","added_by":"auto","created_at":"2024-03-18 12:08:41","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":82043,"visible":true,"origin":"","legend":"\u003cp\u003eMethodology\u003c/p\u003e","description":"","filename":"image5.png","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/b2c3349d1c977db560541d81.png"},{"id":52893094,"identity":"0cd00f1c-788a-45f6-a72d-eb84a857ae42","added_by":"auto","created_at":"2024-03-18 12:08:42","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":68435,"visible":true,"origin":"","legend":"\u003cp\u003eRMSE and RMSE (lagged) for the three input combinations – RMSE and RMSE (lagged) for each model show the lag correlation when lagging the prediction series by 1 day\u003c/p\u003e","description":"","filename":"image6.png","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/ca33405a4e56919dd0d39109.png"},{"id":52893502,"identity":"f84b3f3b-a783-48f0-ac8d-a08407fdf4c0","added_by":"auto","created_at":"2024-03-18 12:16:42","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":503295,"visible":true,"origin":"","legend":"\u003cp\u003eThe predicted and observed water levels of Mahaweli River for the training period\u003c/p\u003e","description":"","filename":"image7.png","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/492f78abe377be602f02908b.png"},{"id":52893501,"identity":"b789d17f-844e-475b-8ea1-f36e99d163c6","added_by":"auto","created_at":"2024-03-18 12:16:41","extension":"png","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":216583,"visible":true,"origin":"","legend":"\u003cp\u003eScatter plots of predicted and observed water levels during the training period, with NSE representing the Nash Sutcliffe Efficiency\u003c/p\u003e","description":"","filename":"image8.png","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/39e8df355bd1f90baef9ca15.png"},{"id":52893095,"identity":"9eb353ab-9d9e-4fdb-a17d-b36efab83604","added_by":"auto","created_at":"2024-03-18 12:08:42","extension":"png","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":482929,"visible":true,"origin":"","legend":"\u003cp\u003eThe predicted and observed water levels of Mahaweli River for the testing period\u003c/p\u003e","description":"","filename":"image9.png","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/7bd2413bae5b7bdbfc7fa742.png"},{"id":52893097,"identity":"9f4970bb-9850-439e-ab3c-7e7c3c0930a2","added_by":"auto","created_at":"2024-03-18 12:08:42","extension":"png","order_by":11,"title":"Figure 11","display":"","copyAsset":false,"role":"figure","size":200031,"visible":true,"origin":"","legend":"\u003cp\u003eScatter plots of predicted and observed water levels during the testing period, with NSE representing the Nash Sutcliffe Efficiency\u003c/p\u003e","description":"","filename":"image10.png","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/2329da83817c29147fffb98b.png"},{"id":52894021,"identity":"1b58e549-e753-49cf-b64b-626e97316c4d","added_by":"auto","created_at":"2024-03-18 12:24:44","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2947192,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4115691/v1/39a71972-cb1c-4367-b2c6-e7202973f06f.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eBehavior of LSTM and Transformer Deep Learning Models in Flood Simulation Considering South Asian Tropical Climate\u003c/strong\u003e\u003c/p\u003e","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eNatural catastrophise such as hurricane, earthquake, and floods lead to significant economic, ecological, and social damages and casualties. Among them, flood can be identified as a phenomenon with severe effects, impacting about 109\u0026nbsp;million people throughout the world between 1995 and 2015 (Alfieri et al., \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2017\u003c/span\u003e; Hirabayashi et al., \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e2013\u003c/span\u003e). As percentages, about 55% of people and 43% of all events were impacted by floods with lost assets totaling over 636\u0026nbsp;billion USD(Serinaldi et al., \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e2018\u003c/span\u003e) encouraging researchers worldwide to mitigate this disaster (Hallegatte et al., \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). When considering South Asian Tropical regions, which are highlighted due to rapid occurrence of seasonal reversal of the wind direction accompanied by intense precipitation, and resultant wet summers and dry winters (Xie \u0026amp; Saiki, \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e1999\u003c/span\u003e), large-scale floods and droughts can be expected (Parthasarathy \u0026amp; Mooley, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e1978\u003c/span\u003e). One of viable option in flood hazard management is practical and effective flood warning systems (Boulange et al., \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). Early prediction of floods facilitates timely management of hydro-junction operations and fast evacuation of individuals from flood-affected regions, leading to a reduction in socioeconomic losses (Zhang et al., \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e2022\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eA significant challenge in advancing flood forecasting technology is the limited availability of field data. Usually, flood prediction approaches can be divided into two categories, physically based models, and data-driven models. Physical models(Mourato et al., \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2021\u003c/span\u003e; Pierini et al., \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e2014\u003c/span\u003e) often require substantial amount of both hydrological and geomorphological data for calibration and validation, and they might not always be readily accessible. Furthermore, the model parameters must be carefully tested and evaluated, because they are regionally dependent and can be challenging to estimate.\u003c/p\u003e \u003cp\u003eTo overcome these limitations, machine learning based data-driven models have gained popularity in flood forecasting because of their ability to capture complex nonlinear patterns, cope with limited data effectively (Rahmati \u0026amp; Pourghasemi, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e2017\u003c/span\u003e), and ability to capture spatial data from images (Lee et al., \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e1990\u003c/span\u003e). These models can be effectively implemented solely based on available rainfall data and measured discharge data, without the need for detailed catchment characteristics. Artificial Neural Network (ANN) is a common algorithm for flood simulation because it has outperformed traditional methods on many occasions (Chu et al., \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2020\u003c/span\u003e; Elsafi, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2014\u003c/span\u003e; Tamiru \u0026amp; Dinka, \u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). Then, Recurrent Neural Network (RNN) was introduced for time series forecasting tasks with the ability of capturing essential information from long sequences of data. LSTM, which is a special type of RNN, have gained significant popularity and widespread adoption in hydrologic prediction tasks (Dtissibe et al., \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2024\u003c/span\u003e; Fang et al., \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e2021\u003c/span\u003e; Xiang et al., \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e2020\u003c/span\u003e; Zou et al., \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e2023\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eAs a solution for some limitations of these traditional neural network algorithms such as low computational speed and ineffectiveness of capturing long-term dependencies, google initiated a new architecture called Transformers(Vaswani et al., \u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e2017\u003c/span\u003e) which is based on attention mechanism (Bahdanau et al., \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2014\u003c/span\u003e). Although this was originally designed for natural language processing (NLP), the Transformer model has denoted its effectiveness in handling other types of time series data (Farsani \u0026amp; Pazouki, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2021\u003c/span\u003e; Wu et al., \u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). In the realm of flood forecasting, there is a scarcity of studies that incorporate Transformer architecture, representing a notable gap in the literature. Moreover, existing research indicates that the accuracy comparison of models, including Transformers, varies across different datasets (Wei et al., \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e2023\u003c/span\u003e; Xu et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2023\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eWhen considering the Tropical regions, especially areas in South Asia, lack of studies regarding deep learning-based flood simulation is an issue. This paper is related to our study (Madhushanka et al., \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e2024\u003c/span\u003e) which focused the lower reach of the Mahaweli catchment, Sri Lanka. In this paper, apart from the forecasting capabilities of daily stream flows, the effects of different input features on the output are thoroughly investigated.\u003c/p\u003e"},{"header":"2. Literature Review","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\n \u003ch2\u003e2.1. South Asian Tropical Climate\u003c/h2\u003e\n \u003cp\u003eSouth Asia, encompassing Afghanistan, Pakistan, India, Nepal, Bhutan, Bangladesh, Maldives, and Sri Lanka, stands as the world's most populous and agriculture-dependent region. The climate in the South Asian region is characterized by the South Asian Monsoon, a significant seasonal phenomenon marked by dramatic shifts in winds that bring vital rainfall to the area. Unlike regions with consistent precipitation year-round, South Asia experiences distinct dry and wet seasons. The summer monsoon, occurring from June to September, carries moisture-laden winds from the Indian, resulting in heavy rainfall. Conversely, the winter monsoon, from December to February, brings dry continental winds from the north (Xie \u0026amp; Saiki, \u003cspan class=\"CitationRef\"\u003e1999\u003c/span\u003e). Additionally, there are two inter-monsoon seasons, the first inter-monsoon from March to May and the second inter-monsoon from October to November (Wickramagamage, \u003cspan class=\"CitationRef\"\u003e2016\u003c/span\u003e).\u003c/p\u003e\n \u003cp\u003eSouth Asian countries are particularly susceptible to temperature and precipitation extremes, including floods and droughts, due to the effects of global warming (Naveendrakumar et al., \u003cspan class=\"CitationRef\"\u003e2019\u003c/span\u003e). The frequency of intense precipitation events with the potential for extreme outcomes is projected to rise across various regions in South Asian countries (Christensen et al., \u003cspan class=\"CitationRef\"\u003e2007\u003c/span\u003e) while Central Asia is expected to have less rainfall compared to the past (Donat et al., \u003cspan class=\"CitationRef\"\u003e2016\u003c/span\u003e). Given these factors, there is an urgent need to establish a reliable flood forecasting procedure to prevent future catastrophes.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\n \u003ch2\u003e2.2. Mahaweli Catchment\u003c/h2\u003e\n \u003cp\u003eThe Mahaweli River, the longest river in Sri Lanka, stretches 335 km in length, originating from the central hills of the country as a collection of numerous small creeks. It traverses through the central region of Sri Lanka before reaching its terminus at the southwestern side of Trincomalee Bay, where it merges with the Bay of Bengal. The Mahaweli River Basin (MRB) is the largest river basin in Sri Lanka, covering an area of approximately 10,448 km\u003csup\u003e2\u003c/sup\u003e, which represents about 16% of the country's total land area (Diyabalanage et al., \u003cspan class=\"CitationRef\"\u003e2016\u003c/span\u003e). The runoff from the Mahaweli River contributes to one-seventh of the total runoff of all rivers in Sri Lanka, with an average annual runoff of 8.8 × 10\u003csup\u003e9\u003c/sup\u003e m\u003csup\u003e3\u003c/sup\u003e (De, \u003cspan class=\"CitationRef\"\u003e1997\u003c/span\u003e)\u003c/p\u003e\n \u003cp\u003eThe distribution of rainfall within the Mahaweli River Basin is uneven both spatially and temporally due to its topographical features. The MRB can be divided into two main parts based on topography: the Upper Mahaweli Basin (UMB) and the Lower Mahaweli Basin (LMB) (Hewawasam, \u003cspan class=\"CitationRef\"\u003e2010\u003c/span\u003e). The UMB, situated in the western part of the central highlands, experiences a total annual precipitation of around 6,000 mm (Zubair, \u003cspan class=\"CitationRef\"\u003e2003\u003c/span\u003e). Conversely, most parts of the LMB are classified as dry regions, such as the North Central and Eastern provinces, with mean annual precipitation ranging from about 1,600 to 1,900 mm. The precipitation in the UMB is primarily influenced by the southwest monsoon, while the precipitation in the LMB is affected by the northeast monsoon, owing to the intricate terrain and monsoon patterns in Sri Lanka (Shelton \u0026amp; Lin, \u003cspan class=\"CitationRef\"\u003e2019\u003c/span\u003e). We selected the LMB as our study area because the climatic data of the region align well with the climatic characteristics of the South Asian Tropical zone. This choice ensures that the case study accurately reflects the conditions typically observed in this region, enhancing the relevance and applicability of our research findings.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e\n \u003ch2\u003e2.3. Long Short-Term Memory (LSTM)\u003c/h2\u003e\n \u003cp\u003eThe Long Short-Term Memory (LSTM) architecture shown in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e, was initiated by (Hochreiter \u0026amp; Schmidhuber, \u003cspan class=\"CitationRef\"\u003e1997\u003c/span\u003e) as an upgraded version of recurrent neural network (RNN) for addressing the limitation of traditional RNNs in capturing long-term dependencies (Cho et al., \u003cspan class=\"CitationRef\"\u003e2014\u003c/span\u003e). Unlike RNNs, LSTM incorporates an additional cell state or cell memory (c\u003csub\u003et\u003c/sub\u003e) where information can be stored along with gates (represented by dashed rectangles in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e). LSTM cells consist of three gates with different functions, namely the forget gate, the update gate, and the output gate. The forget gate, denoted by red rectangle, determines the extent to which elements of the previous cell state vector (c\u003csub\u003et−1\u003c/sub\u003e) will be forgotten. The update gate (green rectangle) determines the extent to which the information from the current input x\u003csub\u003et\u003c/sub\u003e is utilized for updating the cell state (c\u003csub\u003et\u003c/sub\u003e) and then the output gate (blue rectangle) manages the information from the cell state (c\u003csub\u003et\u003c/sub\u003e) which moves into the new hidden state (h\u003csub\u003et\u003c/sub\u003e).\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\n \u003ch2\u003e2.4. Transformer and Attention Mechanism\u003c/h2\u003e\n \u003cp\u003eThe Transformer, denoted in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e, is a special type of neural network architecture that was initiated by (Vaswani et al., \u003cspan class=\"CitationRef\"\u003e2017\u003c/span\u003e) and it has been a game-changer in the field of natural language processing, particularly for solving machine translation tasks. However, its innovative design has found applications in various other domains, including those that involve the analysis of lengthy input sequences, such as time series forecasting and classification, as highlighted by (S. Li et al., 2019).\u003c/p\u003e\n \u003cp\u003eOne of the key features that set the Transformer apart is its self-attention mechanism which replaces traditional recurrent layers in sequence analysis. As the first step of the process, each element in the input sequence is transformed into Query (Q), Key (K), and Value (V) vectors, with a dimension of d\u003csub\u003emodel\u003c/sub\u003e. Then, the Q vectors are matched against the K vectors by performing dot-product multiplications between each pair of Q and K vectors generating a square matrix where each element represents the relationship between the corresponding Q and K vectors. This matrix is then scaled by dividing the square root of the dimension of the Key vectors (d\u003csub\u003emodel\u003c/sub\u003e) and the scaled matrix is passed through a softmax function to compute the attention scores, which represent the importance or weight of each element in the sequence concerning all other elements. These attention scores are generated for each element in the sequence and the obtained attention scores are multiplied by the corresponding Value (V) vectors to produce a new representation of the input sequence. These separate attention heads are concatenated to generate a large matrix, which is known as Multi-Head Attention (MHA). Finally, the concatenated representations are passed through a final linear transformation. The following Eq.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e summarizes the self-attention mechanism.\u003c/p\u003e\n \u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\n \u003cdiv id=\"FileID_Equ1\" class=\"mathdisplay\"\u003e$$\\text{A}\\text{t}\\text{t}\\text{e}\\text{n}\\text{t}\\text{i}\\text{o}\\text{n}\\left(\\text{Q}, \\text{K}, \\text{V} \\right)=\\text{s}\\text{o}\\text{f}\\text{t}\\text{m}\\text{a}\\text{x}\\left(\\frac{\\text{Q}{K}^{T}}{\\sqrt{{d}_{model}}}\\right)V$$\u003c/div\u003e\n \u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\n \u003c/div\u003e\n \u003cp\u003eAdditionally, positional information of elements in the sequence is achieved through static positional encodings using Sine and Cosine functions, and then they are added to the original input embeddings. The original transformer consists of an encoder decoder architecture as depicted in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e. In the decoder, a casual mask is used when calculating the self-attention, preventing each token to attend their future ones. In contrast to self-attention, cross-attention, also known as \"encoder-decoder attention,\" is used to capture relationships between tokens in different input sequences. The output of the encoder is transformed into the query and the key matrices, and the output of the self-attention block of the decoder is transformed into the key matrix in order to calculate the attention as mentioned before.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e\n \u003ch2\u003e2.5. Evaluation Metrics\u003c/h2\u003e\n \u003cp\u003eThe following indicators (Eq.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e–\u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e) were used to quantify the accuracy of the models. They are Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), Nash-Sutcliffe efficiency (NSE) and Coefficient of Determination (R\u003csup\u003e2\u003c/sup\u003e). It is preferable for the values of NSE and R\u003csup\u003e2\u003c/sup\u003e to be close to 1 while values of RMSE and MAPE close to 0 are considered desirable. The above indicators were calculated using the following formulas.\u003c/p\u003e\n \u003cdiv id=\"Equ2\" class=\"Equation\"\u003e\n \u003cdiv id=\"FileID_Equ2\" class=\"mathdisplay\"\u003e$$RMSE= \\sqrt{\\frac{1}{n}\\sum _{i=1}^{n}{({X}_{obs}^{i}-{X}_{sim}^{i})}^{2}}$$\u003c/div\u003e\n \u003cdiv class=\"EquationNumber\"\u003e2\u003c/div\u003e\n \u003c/div\u003e\n \u003cdiv id=\"Equ3\" class=\"Equation\"\u003e\n \u003cdiv id=\"FileID_Equ3\" class=\"mathdisplay\"\u003e$$MAPE= \\frac{1}{n}\\sum _{i=1}^{n}\\left(\\frac{|{X}_{sim}^{i}-{X}_{obs}^{i}|}{\\left|{X}_{obs}^{i}\\right|}\\right)$$\u003c/div\u003e\n \u003cdiv class=\"EquationNumber\"\u003e3\u003c/div\u003e\n \u003c/div\u003e\n \u003cdiv id=\"Equ4\" class=\"Equation\"\u003e\n \u003cdiv id=\"FileID_Equ4\" class=\"mathdisplay\"\u003e$$NSE=1-\\frac{\\sum _{i=1}^{n}({{X}_{obs}^{i}- {X}_{sim}^{i})}^{2}}{\\sum _{i=1}^{n}({{X}_{obs}^{i}- \\stackrel{-}{{X}_{obs}} )}^{2}}$$\u003c/div\u003e\n \u003cdiv class=\"EquationNumber\"\u003e4\u003c/div\u003e\n \u003c/div\u003e\n \u003cp\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAiMAAABSCAYAAACGy9T8AAAWD0lEQVR4Ae2d0ZGmthKFNwXH4BQ2B4ewz350CpuBM3AGjsAROAEn4Aycw9z6xj51e3okkAAh+DmqYgEhdbeOmu6D4J/98uZiBIyAETACRsAIGIGJCHyZqNuqjYARMAK3QOCff/55+/XXX99+/PHHty9fvrx9/fr17a+//rqF7TbSCNwBAZORO8ySbTQCRmAqAhCRb9++vUFK2CAlnLsYASNwDAImI8fgaClGwAg8CIHv37+//fLLLw8asYdqBMYiYDIyFl9LNwJG4MUQ0MrI33///WIj83CMwDwEHk9GeP/L5mIEjIARAAFeyRAT/vzzz0+AQER++uknfy/yCRlXGIF9CDw+C5uM7HMg9zYCr4ZAjYyYiLzaTHs8V0LAZMQrI1fyR9tiBKYjUCIjmYhw7m9Gpk+VDXghBExGTEZeyJ09FCOwH4ESGVGdVlLZ87rGxQgYgWMQMBkxGTnGkyzFCLwIAiIepW9GXmSIHoYRuBwCJiMmI5dzShtkBGYiYDIyE33rfioCJiMmI0/1fY/bCBQRMBkpwuJKIzAUAZMRk5GhDmbhRuBuCJiM3G3GbO8rIGAy8mAy4v9b4+MtDB78SmJEMdZlVEdiXta4Xmsyso6RWxiBoxEwGXkoGeFnib///vvR/nRreSRGcDmakBjruluMwryucf2Kycg6Rm5hBI5GwGTkgWSEYMvm8hkBkuORP9k01p8xzjVHY57l956bjPQi5vZGYD8CJiMPIyP8fxr8j6NbCklja98t+mb1OYpA7MGasf/www9vT/n/T47C/AifMRk5AkXLMAJ9CJiMPIyM8Mrgt99+6/OSSa3jH5gafUwCUuE1DURg7+uaO2GtsY/GOcsX7kdhrnHs2ZuM7EHPfY3ANgRMRi5GRviOg0RIQPz27dv7f9h1FHnYE/D5A1AkkqNs2eau5/UC+z3f1OzBmlGCNTY8qezF/CisTEaOQtJyjEA7AiYjJ5IRBbn8dKhzEv4ff/zxTkR4HcJrEfqwbSmSq28gkP3169ctot77POl/K4V07SEDe7Fmzp9C/OSQEXN8Tf579l9C1X16tl7h4L0ReCICJiMnkhHe/7Pq8f379w++Bukg8Cr4xV9fcExi21KQGQtBFnmlAkmhPfZRFJCjjFnfi7TYVhrTnjrmQlhskbOENfMPrmzMPbp0Lh+ACHFtRsHfZI8Ikc4Z16hSwhy9wmSU3ixXvn+23myHz43AkxD4mK2eNPL/xqoge9bQCe6lAEvyUfAj6evDRRIiS/661mMnemLhabOWTESIRHzYa3UGGeinP3vZFmWPPF6zbYRuMI/4Me4SkaOuhMcS1sgGW5FSvdKJr4XQjdwt834EHvhjXEXjWPYeIb8kI2NOm9K9Uup7ZJ3JyJFoWpYRaEPgY7Zq6/NSrQh2MemcMTgCewz0UScJKK5AcExbAnVvyeMiQepJtySLxMom4hF1ihCMTkglu6hbsq3WZ299ToSyQXI5jwRC9ezXsKYfc8t80zavgkBCqY9zEOWPPsYuxo9dS+M82o6MeT4/Wl9JnslICRXXGYGxCJiMTCAjSuy1VYqjppxAHgvJbelJWwloZhKM9sbjGbaVEqEIifbRxni8hjVtISNsmYhEOTOPGSOkqEa4RtiWMc/nI3RmmSYjGRGfG4HxCHzMVuP1XU4DwY7t7KKAR5IdVfK41hIkSUdP5KNswgZhnvdcq5Wttinh1+Qu1WNfiby1yFzDGgKCHL2GW7JjzzX5WcZa5zXZrMhorkoY1Prtrc+Y5/O98lv6C7Mzx91il9sYgVdG4PwsfDE014LyKHMhISSiEhlRotqrm7HFQnLRNyGxnmM96ZOEsKs1ELck5qyr93yrbehhHFt/FVNKhNjCqyrZVBvLEtYQK716ox3Jb60c5RNrerguXeyxDRvPKhnzfH6GHSYjZ6BsHUbgIwIfs9XHa484I9ixnV1IZkvfb/TYQ3KDFOSSx1VKfJAPJVj1p11rAtqT7KWvtm+1jaSJvYwXHCLBI7EwPq1C6JoSLn1q3+9wTe2xETlgrcI5W6mUsKYdcx71Yd/o1ZGSfbU65jPawzk4sFeBjFHHBrlVG/mzVrE4hwjSjjodq53kxT1tI+acR92x7ahjk5FRyFquEagjcH4Wrtsy5QrBju3MosBc0pkDu5Ks7NReAZrArj5ZXh4XiZNAGwt1tCMBUSSLOrUVKVC7uLpCG2TkZE8f2Y7smGCi/qXjFtvoT3KXrSRKHXMNG5CjMegadcyD6rMdkJWIH7jQPhfkaC7iNeqlS/X0RyYb8ik6x05KCTfNiZK4yADzwNjBV8dR9rvAjn+YI9kjXZpD6im0EfGV71FPO40JWxg77ajLxxkXmZgxpx69JXzVZ8Qe+2boHTEWyzQCd0HglCxMECZocoMToEpBfRZg2MR2ViGYgwVJJxYCoJJ8DOyxzdJxaQy5juShpLckK18j+fFUi83YGGVwXEr2zDH1lK2vSbIdpXMSmJIj1zMZiU/52CHfox+2s+W5QI6Se0lnS91WrGu4YSc2y7Zoez4enbxFLoQlNsU54Jy513WORW44lp+/Dyb8U8IcHx49nmDC+6HJSEbE50ZgPALDszCBhEClJ2MSgp7Cxw9vXQPBLift9V7bW0BEcnAlGYKR6nNgb9FWGkOuI5ltwZ4+mr9MRmrJnjGReBiLkmjLOHrbgBmYUjiOtsZrJEbZyhiUHEnkwj3qhtSwbS1bsa7hln2Cc81JxJjjErnaOo7cT6QUzMCOomP20aZ4LB/QHLRijg+X2ma7jjw3GTkSTcvaggD3sO6vtf7cH/js3ctmMgIABIq8kRhqTz6AxbUcWGeCKPvPsIHkJn2lPZiyKUESzDmutZXNOC5tchJSv+jUMXGp/9pesrGH+dUTL7aKCMRkjx1aDeEmGX2jCCPGFn0PvdiH/XHc2BqxybiBR2y/hk/teq+MGm7ZJ5gHZFM4JsFTSPhgwZ76EQW9YIdO+QH6qBN5q9mHPfRnTtYw15wil/GfWfCbGXrPHKN1XRcB7ifukxjLsFZ+iW+yKfZyjQe/2n113ZF+tGwzGUGMkquCBQEGUAAKQEuFIJNBLrU7q04Te5a+NT05sK+1zw7K+VIhgTBHPUU6SEBRPselZM8YaAu23FQ1X+ix4cy2Sv57dfZiXcMt+0SUy7GIn/r3zu/ecR7R/yjMj7BF/q64doRMyzACLQiQQ2sxk3rlK/Za3ZVc7vs73vuyfxcZ0U0rYey5gQEqJi1dByiC55WKJvdKNo22BQIx6sl5tO1nyIcwH5WIjHXbjB2JeZvGeivFtaN8oK7JV4zARwTwvRKhiA8gH3t8PIOw3NVvd5ERAghbLDUyckUigt1PJCM8QTNvpaXyOJdPPOZpo0Skt2JhrNeROxrzdY3LLUxGlvHx1TEIEI9ZUSZm5ALJYCOPLj1I4rs5J2dZVz3fRUZI5HpPrAHqRo6AZSKiZWX1mbl/IhkBbxy+xMBnzsVs3bw+zEufR9hkrOsojsK8rnH9imLYXZ8w10foFldEgNUP8lEuesBXrmJfe8OgtjH/ZnlXPf888kZLNWh9/8HgdRNHoNQuAlkCPKvN7ZfOc9+ec8nt6eO2RsAIvC4CimPELhcjcBYCPBwurWrgj7RRziqtoLC6wvWYg8+yf6+ezWREN6yAYQ+Qd7uBZX8GUvXef/7FlDExJq/oA4oBim13i2Wy3/t7IkD+XCIjGhULANx/tZVtruHDdyubyQig8X5LhXM+1rtbUVC9m9221wgYgTEImIyMwdVSlxFoJSNIWVpFeRwZycxMbE2vbZZhX78qktCyX5dWbyH59Ra+YgSMwJMQMBl50mxfZ6w9ZITXMF4ZCT/fje+l+GaExH635SGTkevcjLbECFwBAZORK8zC82yAXLS+XeCHI7XXiOS0mJvvguSm1zT8GoYB51UQfnrUCuZVANpKRmCx6uu9v6GwD9zbB2I8MhmJaPj4LAQgEMSRXKhXruUDVX7xV1sVgaAg4xG/ptGNquAbv+gFIOprQGWQr3CucfTaEr+X6e3r9kbACFwXAcW42pPndS23ZXdGQH9nJPud/JFcxcO+iElprBCVuy0IaByfaZiuPGS/lYywMuJiBIzA6yGg4J+TwuuN1CO6GgL43p6Hed5OLJGVq4032mMysuF/7YXBmoxEN/KxEXgdBExGXmcu7zYScgsrG1u++WBV5Ep/ULQXe5ORDWSEJ6Y97LV3ktzeCBiB8xAwGTkPa2v6jACfPvAZQA8hwWchMZCZuxaTkY1khMl3uS4CfMClD615Fef5uu5cjbaMhwcCNX5AkF9bxjYZGT0jlr+GAKSidZUD/+bXNXcmIuBhMrKBjLAc5uS2djvNva4ExA1K8vFrtbnzMVM796pWMgnaa/euycjM2bLupyJgMrKBjBCsSHYu90AA8tiz5HmPUdnKLQhASuIvAEsyTEZKqLjOCIxFwGRkAxlZ+oMzY6drm3SWpu/4u/Nto/3Yy0TkXzxY8n0yIWOFjPt2jYiAlsnIx3vIZ0bgDARMRjaQEZb8Z7yf03cQV1+V0c+lz9j//PPP1felJF8lYC3Tn3FTjdRxFx8AgzPmP+pY8oVIRNZ8wWRkpAdbthEoI2AyspGMlOH8XAtpgbzowzn2bL0FOfyGvOXJLsomWLd+CBX73ekYTDJBJGnHRMXxzEIyxAbIkT6m7J3LrT4g/5s5/rN0l3wBzKMvgMdSMRlZQsfXjMAYBOZG6DFj6pKqINXTiWRCUdCSjLxnBYOkSIDkGgmJwnFv4WlOT/k9fbGRVxWvXhgn29mlxQewSR/RijSx713h2uoD6Hl1Qhrnfa8vaE575yfa4GMjYAT6EOjPin3yL99aBKLHUD1ZQTRY5RDJkAyRDwUz2rGqQeFYZEbt1/bSs9audJ0k1PsEXpJz9TpWDZgL9mcWzc2aD2ATPkB7HffYKj1bxgYZfQIhFTZ7fcFkREh6bwTOQ8BkZOdrGoI8hEbEQ1MHCVAdKxp6T017gt3a3zqQHPb0KT3Zaukf/RAO9IlcSTfnJDKdR7kjj1tsO1o/JK8H16P0t/gAcyBCylxBaNmLnKzZUvMBxqs5pw1F5/gZBV20m4ENxED2yIexhzqR+ncjD/5njy+YjBw8GRZnBBoQMBnpJCMk9fwUTOBbWu2Iy+t6f93zyoUgrsQS55RAT4KTPXoijLJZLSDoc+3M0mLb0faAQwmnVj3MrUhj7EPdGmlY84FISCEhzEtJV9Qbj2s+QBuuRf/jWD6h6/gJemcUkTX5ILaN9sk9vmAyMsNLrPPpCJiMbCAjOeER5HnSy/VHOReBOxKMKJd6Lf/TblbCiTbp+GzbwB8M9hQIQiQJHNewj3pm+gBECf/DhlZ7o+1nHOOj4AgxifiO0r3HF+gLnmevJo7CwnKNwB0QMBnpJCMKqHlyFcDWnqBzv5ZzEuxSYCTQz3zyXRrDFttIViSDXiz3JKA4BvTHLV5bOp7pA9jLaksLcVoaw6hr2MWcxhWbUbqQu8cXNI9L99xI2y3bCDwRAZORTjJCoCoFKRInyaA3gbY43RIZ4WmYhI9uLYO3yOxpo+BMMiltNVl7bGM8vWVPAsq6RKJy/dL5LB9g3vER5qbkm0s291xDdmn+VbekW2SJOTqj7PEF+fvSeM4Yg3UYgSchYDJyEBkh2OoDwpID8V6/56k1EhsSTakvdXwfoGTUGuj1lFqSWbJ9S12LbdjLOElm0XaSAGNWgtWHlxon7SM+sg8Z9NlbmEue4NmztZY1H+h9RRF9puYDInzse8Zfwq91nD3tmDN8FH9gXtFLnQr18gGtmjBW6igaH1jo2xPhjB9QXyo9WOT+9EW2yUhGxudGYBwCJiOdZIRAGYMpU0NArQXFI6YO2QTIWAjM8aNFrudAH9vnY8ZBoB9RWmyDYLD6wGoCeBL8VRgLY+Ma2GIrhWORgxLeez5alG7ko0eFc+lUXWk/wwdycuf8SkmU+WMeRSbBDfuiL8sHRKZowzg0vxzTH98GY41Zx/KNPCd7fAH7roRjHpvPjcArIvD/DPCKo2sYE0GHrbXk4KeAmwkKAU1BmD560kOPdOa9bKBewZg6kns8JxCrrwiFzqN92IBersUEgEzqCdhc0xMp9aqjXvbLrpZ9q22Mh7aUTEYYA0mHwthFBmjHMQlM435v9N8/OfHFay3H6JRNsT06ZU+s13GLDyjByW7GqDmLe+nJPpN9AJ3qxzVKlCnbSvNJP/kT48UXJJ9r1HGdY8mWvJ498xR1CQPqNE58jHbRB9EZ9WKPfEC2YQfHsV+0bY8vyE7ZGOX62AgYgTEItGfhMfqnSyUwsuVC4iMoKWjrOkEuFs5z0KIvAVb17LOcKCMfozcGYxJPJDO5felcQZ6+OdljD/LYK6khg2PspqBf9pfk760jcWKjCIaSDXKZD+pJ3LKTc2EIPmyxcJ227M8uLT6ATT1zmH1miw/U5jP6F3PAOfMO3vk443wkthAJdGruJFu+gW0UfAPioWPdG/gDOKnde4P/yO0eX2DMkTBJrvdGwAiMQ+BzFh6n65KSa2SE4Eegy0GJQKkSnzolJ+6VzAmeCqDqu7QnyOoJWu1iQFbd0h4ZCuAEe+xSIdgqyaBHBITr1HOuvupz9J4EQsLALsYmEgFmJHfquS47sFPtlTijTXFMsX70casPYL/IVItNJZ/p9QH0lOYz+1eUy7F8leOc6Ftsb22DfN0vcYUD++IccwyxosR6+uMT+V7Z6wv0xy7dv63jcTsjYAS2I/D/DLVdxq17KhjWBkFgiqshkYzU+uT6/KQnnXmvftQTfGMwJFnHQKy2tb100o+gzaYSrzE2xkghCdAevVvGKfln70lG2CtCc7b+Fn1KkCJX2Jvnn3PNueYokoFeH6jNZ/avWrIn0csfWsZ4hTZH+ILJyBVm0jY8DQGTkcprGjkCwVhP6CSK+ASnNmv7/KS31h59peQKQUGWEtaSnNrKA31IPuggKcXxUE+dxrsk/yrXGCdjuDIRAStWG3pwrflMjw/U5jP6F/JoR+GYayrUQ1avjq3sPcoXTEaEqPdG4DwETEZWyAhTQSIhMBPstIpw3hRZkxEwAmciYDJyJtrWZQT+RcBkpIGMAJWeEvU+3Q5kBIzAayJgMvKa8+pRXRsBk5FGMsKrkfhO/9rTauuMgBHYioDJyFbk3M8IbEfAZKSRjAAx33G0fK+xfTrc0wgYgdkImIzMngHrfyICJiMdZOSJDuIxG4GnIWAy8rQZ93ivgIDJyH9khFcw2q4wMbbBCBiB8xAQAVEM8CvZ87C3JiMAAo8nI3YDI2AEjIARMAJGYC4CJiNz8bd2I2AEjIARMAKPR8Bk5PEuYACMgBEwAkbACMxFwGRkLv7WbgSMgBEwAkbg8QiYjDzeBQyAETACRsAIGIG5CJiMzMXf2o2AETACRsAIPB4Bk5HHu4ABMAJGwAgYASMwFwGTkbn4W7sRMAJGwAgYgccjYDLyeBcwAEbACBgBI2AE5iJgMjIXf2s3AkbACBgBI/B4BExGHu8CBsAIGAEjYASMwFwE/gdV6eH2EG2yUgAAAABJRU5ErkJggg==\"\u003e\u003cbr\u003e\u003c/p\u003e\n \u003cp\u003ewhere \u003cem\u003en\u003c/em\u003e denotes the number of the observations; \u003cem\u003eX\u003c/em\u003e \u003csup\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sup\u003e \u003csub\u003e\u003cem\u003eobs\u003c/em\u003e\u003c/sub\u003e and \u003cem\u003eX\u003c/em\u003e \u003csup\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sup\u003e \u003csub\u003e\u003cem\u003esim\u003c/em\u003e\u003c/sub\u003e represent the observation and simulation on day \u003cem\u003ei\u003c/em\u003e, while \u003cem\u003eX¯\u003c/em\u003e \u003csub\u003e\u003cem\u003eobs\u003c/em\u003e\u003c/sub\u003e and \u003cem\u003eX¯\u003c/em\u003e \u003csub\u003e\u003cem\u003eobs\u003c/em\u003e\u003c/sub\u003e denote the mean values of the observation and simulation series.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\n \u003ch2\u003e2.6. Lag Correlation\u003c/h2\u003e\n \u003cp\u003eIn hydrology, the timing misalignment between predicted and observed data is an issue. This temporal discrepancy can be quantified and analyzed using lag correlation metrics, providing insights into the timing errors. The process involves computing a selected error matrix (e.g., root mean square error), then systematically adjusting or lagging one of the time series relative to the other and recomputing the error matrix (Jackson et al., \u003cspan class=\"CitationRef\"\u003e2019\u003c/span\u003e). This approach indicates the time lag at which the correlation or similarity is maximized between observations and predictions. Hyndman \u0026amp; Khandakar (\u003cspan class=\"CitationRef\"\u003e2008\u003c/span\u003e) utilized lag correlation measures to gain insights into their dataset's ability to capture specific events, despite the timing variations.\u003c/p\u003e\n \u003cp\u003eIn our case, we used RMSE value to determine lag correlation between simulations and observations with 1 day lag time. We calculated the percentage deviation of RMSE between the original and the lagged prediction series in order to quantify the lag correlation.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"3. Methodology","content":"\u003cdiv id=\"Sec10\" class=\"Section2\"\u003e\n\u003ch2\u003e3.1. Study Area\u003c/h2\u003e\n\u003cp\u003eThe Somawathi area, located downstream of the Parakrama Samudra reservoir in the Polonnaruwa district and within the Lower Mahaweli Basin (LMB), is recognized as a flood-prone region. Floods in this area typically occur between December and February, coinciding with the northeast monsoon (NEM) season. To monitor flood levels effectively, the Manampitiya river gauge station plays a crucial role. Real-time water level data, along with alert levels, minor flood levels, and major flood levels for the station, are available on the Irrigation Department of Sri Lanka's website. Figure\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e shows the map of the study area.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\n\u003ch2\u003e3.2. The Dataset\u003c/h2\u003e\n\u003cp\u003eThe dataset pertaining to the area comprises daily water levels (in meters) recorded at the Manampitiya station and daily rainfall measurements (in millimeters) obtained from three upstream meteorological stations, namely Angamedilla, Aralaganwila and Polonnaruwa Agri. Figure\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e shows the water level and rainfall data while Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e exhibits the statistics of them.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003ctable id=\"Tab1\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003eDataset summary\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth colspan=\"5\" align=\"left\"\u003e\n\u003cp\u003eWater Level (m) and Rainfall (mm)\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003cth align=\"left\"\u003e\u0026nbsp;\u003c/th\u003e\n\u003cth colspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eStation Name\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003cth align=\"left\"\u003e\u0026nbsp;\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eAralaganwila\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eAngamedilla\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003ePolonnaruwa Agri\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eManampitiya\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003ecount\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e10319\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e10319\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e10319\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e10319\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003emean\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e4.94584\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e4.820227\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e4.20901\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e33.382384\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003estd\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e15.083687\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e15.451919\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e13.486747\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.643105\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003emin\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e32.196\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003emax\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e225.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e222\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e184\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e37.254333\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003eBased on the boxplots presented in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e, we can discern the influence of the Northeast monsoon on the region. Upon closer examination, it becomes evident that while the maximum precipitation occurs during December to February, there are also instances of extreme rainfall events throughout the year. Although some of these events are classified as outliers, it is not advisable to use preprocessing techniques such as the Interquartile Range (IQR) to remove them, as some of these points might be attributed to unexpected extreme weather conditions. Such occurrences, including sudden storms with heavy rainfall, are common in Sri Lanka due to the influences of the Bay of Bengal. Furthermore, consistent patterns observed in monthly precipitation and water level data indicate a strong correlation between them. This correlation underscores the interdependence of precipitation patterns and water levels in the region, emphasizing the importance of considering both variables in this flood simulation task.\u003c/p\u003e\n\u003cp\u003eAs outlined in our previous paper (Madhushanka et al., \u003cspan class=\"CitationRef\"\u003e2024\u003c/span\u003e), the analysis was conducted using rainfall data from the designated rain gauge stations along with the water level at Manampitiya, based on the Pearson correlation coefficient calculated among the four stations. Because upstream river gauges belong to a region with different geographic and climatic conditions, they were considered not to be used. Prior to their use as inputs and labels for the models, the data underwent normalization using the Standard Scaling method, which involves considering the mean and standard deviation values of each variable. Subsequently, the dataset was split into the training set and the test set. Specifically, the first 70% of the data was allocated for the training set, while the remaining portion was reserved for the test set to evaluate model performance.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e\n\u003ch2\u003e3.3. Experimental Setup\u003c/h2\u003e\n\u003cp\u003eThe work was conducted using the Python programming language, with data preprocessing, management, and visualization carried out using libraries such as NumPy, Pandas, Scikit-learn, Matplotlib, and Seaborn. For deep learning tasks, the TensorFlow framework was employed. Training of the models was conducted on the Google Colab platform, which provides a cloud-based environment for running Python code, particularly well-suited for machine learning and deep learning tasks. Historical data of the three rain gauges and the target river gauge were used as inputs to forecast the following day water level using the sliding window method. Hyperparameters for the models were kept same as in our previous paper (Madhushanka et al., \u003cspan class=\"CitationRef\"\u003e2024\u003c/span\u003e), which were determined by trial-and-error approach. The Transformer architecture employed has been slightly modified from the original implementation presented in (Vaswani et al., \u003cspan class=\"CitationRef\"\u003e2017\u003c/span\u003e). Notably, the original input data were used directly without being mapped to an embedding vector, which was done considering the continuity of the data for this regression task. Additionally, the mask of the self-attention layer in the decoder was omitted, thereby allowing time series data to access their successors. However, other aspects such as positional encoding, the number of layers and attention heads, and the dropout rate were kept consistent with the specifications outlined in the original paper. All the hyperparameters are shown in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e. \u0026ldquo;Early Stopping\u0026rdquo; was used as the regularization for all the LSTM and Transformer models.\u003c/p\u003e\n\u003cp\u003eAs the next task, we studied the contributions from each input features to the final output by considering the following input combinations. Along with the initial 2 models, another 2 LSTMs and 2 Transformer Encoders were utilized for this task with the same model architectures, except the input layer.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ei. Case 1 -\u0026nbsp;\u003c/strong\u003ePast water levels and rainfall data as inputs (Initial models)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eii. Case 2 -\u0026nbsp;\u003c/strong\u003ePast water levels as the only input\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eiii. Case 3 -\u0026nbsp;\u003c/strong\u003ePast rainfall data as the only input\u003c/p\u003e\n\u003cp\u003eBased on the results, Combination 1 was selected as the best scenario for both LSTM and Transformer encoder models. Then we considered two additional Transformer encoder-decoder models with the same hyperparameters as in the encoder (Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e), for a broader comparison.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003ctable id=\"Tab2\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003eThe values of hyperparameters\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eHyperparameter\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eValue\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eBatch Size\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e32\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eSliding window size\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e15\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eNumber of LSTM units in the hidden layer\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e64\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eOptimizer\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eAdam\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eActivation function\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eReLU\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eValidation split\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.1\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eLearning rate\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.001\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"2\" align=\"left\"\u003e\n\u003cp\u003e\u003cem\u003eFor the transformer\u003c/em\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003ed\u003csub\u003emodel\u003c/sub\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e64\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003ed\u003csub\u003eff\u003c/sub\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e192\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eNumber of layers\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e6\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eNumber of heads\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e8\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eDropout rate\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e0.1\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003c/div\u003e"},{"header":"4. Results and Discussion","content":"\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\n\u003ch2\u003e4.1. Impact of Input Features on simulation of daily water level\u003c/h2\u003e\n\u003cp\u003eThree LSTM and Three Transformer models were used to examine the impacts of the 3 different input combinations for prediction. RMSE values of the results were calculated after lagging the prediction series by one day in order to examine the lag correlation between the 2 series as shown in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e.\u003c/p\u003e\n\u003cp\u003eAccording to the results (blue and green bars) in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e7\u003c/span\u003e, Case \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e has the highest error and case \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e denotes the lowest error while the error of Case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e is close to Case \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e, for both LSTM and Transformer algorithms. For the LSTM, RMSE performance was improved by 47% in Case \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e compared to Case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e while 12% from Case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e to Case \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e. For the Transformer, they were 39% and 9% respectively, denoting a similar behavior. This high improvement from Case \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e to Case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e and comparatively small improvement from Case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e to Case \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e indicate a higher impact of the past water level data on the output, among all the input features.\u003c/p\u003e\n\u003cp\u003eWhen considering the LSTM models (blue and orange bars), there is a large reduction of RMSE between the actual and the lagged scenarios in Case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e and \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e while Case \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e does not show a lag correlation. RMSE of the LSTM was dropped by 76% in Case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e and 51% in Case \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e, while the Transformer encoder (green and red bars) had the values of 67% and 46% respectively, indicating the highest percentage reduction for the univariate analysis (Case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e). These percentage values exhibit the impact of the 1 day lagging of the prediction series. This error reduction might happen when the models allocate a higher weightage to the final timestep of the input series, which was generated by the sliding window method. It explains the univariate analysis showing the highest RMSE reduction while Case \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e, which used only rainfalls as inputs, showing no reduction when lagging the prediction series by one day. We can suggest that low lag correlation is better because one can argue that the prediction series is developed by solely shifting the observation series by a time offset. Case \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e showed the lowest RMSE value as well as a low lag correlation compared to Case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e. Therefore, we used past rainfall and water level data for the upcoming models based on these results.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec15\" class=\"Section2\"\u003e\n\u003ch2\u003e4.2. Model performance\u003c/h2\u003e\n\u003cp\u003eFigure \u003cspan class=\"InternalRef\"\u003e8\u003c/span\u003e\u0026ndash;\u003cspan class=\"InternalRef\"\u003e11\u003c/span\u003e denote the daily water level of observation and simulation at Manampitiya station in Mahaweli catchment for the training (1984\u0026ndash;2003) and testing (2004\u0026ndash;2012) periods in the Multivariate Analysis.\u003c/p\u003e\n\u003cp\u003eAs shown in Fig.\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e8\u003c/span\u003e\u0026ndash;\u003cspan class=\"InternalRef\"\u003e11\u003c/span\u003e, all multivariate models performed relatively well in simulating average water levels of Mahaweli River. However, they differ greatly when it comes to simulating both upward and downward peaks in streamflow. It should be noted that both Transformer Encoder Decoder models significantly underestimated the peak water levels, mostly for daily water levels exceeding 35.5 m, while clear overestimations were observed for low water levels, especially for water levels less than 33 m. Among the models tested, LSTM and Transformer encoder showed superior performance in simulating the peak water levels, while the other 2 models performed poorly in this regard.\u003c/p\u003e\n\u003cp\u003eTable\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e exhibits the evaluation indices of the developed models to compare their performances. Four evaluation metrics (RMSE, MAPE, NSE and R\u003csup\u003e2\u003c/sup\u003e) were utilized for measuring the forecasting capabilities of the models. During the training period, RMSE ranged from 18.09 to 23.59, MAPE varied from 0.282\u0026ndash;0.392%, and R\u003csup\u003e2\u003c/sup\u003e varied from 0.8743 to 0.9185. All models had NSE greater than 0.86. LSTM achieved an NSE of 0.9183 and RMSE of 18.09, with R\u003csup\u003e2\u003c/sup\u003e of 0.9185 and a MAPE of 0.282%. For the Transformer Encoder, those values were 0.9010, 19.90, 0.9014 and 0.314%, respectively. Both Transformer Encoder \u0026ndash; Decoder models denoted a poorer performance compared to the others.\u003c/p\u003e\n\u003cp\u003eDuring the testing period, all models demonstrated satisfactory performance in simulating daily water level, with NSE values greater than 0.7768, although the performances are lower compared to the training phase. The LSTM and Transformer Encoder showed similar performances with RMSE, MAPE, NSE and R\u003csup\u003e2\u003c/sup\u003e values of 27.83, 0.449%, 0.7971, 0.8026 and 27.83, 0.471%, 0.7972, 0.7985 respectively. Transformer Enc - WL Dec \u0026ndash; RF and Enc - RF Dec \u0026ndash; WL denoted similar evaluation values with a RMSE of 29.10 and 29.19 and NSE of 0.7782 and 0.7768 respectively. Transformer encoder exhibited improved performance compared to Encoder-Decoder models.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003ctable id=\"Tab3\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003eThe evaluation of daily streamflow simulation\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003ePeriod\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eModel\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eRMSE (cm)\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eMAPE (%)\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eNSE\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eR\u003csup\u003e2\u003c/sup\u003e\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd rowspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eTraining\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eLSTM\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e18.09\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.282\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.9183\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.9185\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTransformer Encoder\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e19.90\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.314\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.9010\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.9014\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTransformer Enc -\u0026gt; WL Dec -\u0026gt; RF\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e21.51\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.338\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.8844\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.8928\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTransformer Enc -\u0026gt; RF Dec -\u0026gt; WL\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e23.59\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.392\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.8609\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.8743\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd rowspan=\"4\" align=\"left\"\u003e\n\u003cp\u003eTesting\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eLSTM\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e27.83\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.449\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.7971\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.8026\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTransformer Encoder\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e27.83\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.471\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.7972\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.7985\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTransformer Enc -\u0026gt; WL Dec -\u0026gt; RF\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e29.10\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.527\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.7782\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.7828\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTransformer Enc -\u0026gt; RF Dec -\u0026gt; WL\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e29.19\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.52\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.7768\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"char\" char=\".\"\u003e\n\u003cp\u003e0.7901\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003eOverall, LSTM and Transformer encoder models exhibited approximately similar performances in daily water level forecasting. Single Encoder model showed the better performance among all the transformer models. When there is an encoder as well as a decoder in a Transformer model, it takes the Query (Q) and the Key (K) vectors from the encoder and the Value (V) vector from the decoder to calculate the cross attention between the two inputs. But when there is just the encoder, all the Q, K and V are taken from a single input series. That might be the reason for its increased accuracy. However, if there are input series with 2 different sequence lengths, an Encoder-Decoder model has to be used for the analysis.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"5. Conclusions and Recommendations","content":"\u003cp\u003eIn this study, four machine learning models (LSTM, Transformer Encoder and Transformer Encoder-Decoder Model 1 and 2) were applied to simulate daily water levels and extremities at Manampitiya river gauge in Mahaweli catchment. The impacts of different inputs on the output were also of interest. The main conclusions of the study are as follows.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cp\u003eUse of past water level data denotes a higher impact on the output compared to the other input features such as rainfall. It is recommended to use many other input features that show higher correlation values with the target, along with the past observations of the output since this strategy increases the model performance as well as decreases the lag correlation.\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eLSTM and Transformer encoder shows similar accuracies in daily water level forecasting and predicting extremities. Although Transformer models tend to show a better performance in streamflow forecasting as in studies of (Castangia et al., \u003cspan class=\"CitationRef\"\u003e2023\u003c/span\u003e; Liu et al., \u003cspan class=\"CitationRef\"\u003e2022\u003c/span\u003e; Xu et al., \u003cspan class=\"CitationRef\"\u003e2023\u003c/span\u003e), lack of sufficient data may cause the performance reduction for the Transformer Encoder model (Wei et al., \u003cspan class=\"CitationRef\"\u003e2023\u003c/span\u003e).\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eIn this case, switching the inputs between the Encoder and Decoder shows similar performance but it might be different in another case. Therefore, it is recommended to do this strategy and check the accuracy when there are input features with 2 different time steps.\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ul\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eG.W.T.I. Madhushanka\u0026nbsp;\u003c/strong\u003eled the development of research methodology, raw data acquisition, code development, data manipulation, training and evaluation of the models, results analysis and writing the original draft. \u003cstrong\u003eM.T.R. Jayasinghe\u003c/strong\u003e led the supervision throughout the study, providing financial support and facilitating data acquisition by signing the agreements. \u003cstrong\u003eR.A. Rajapakse\u003c/strong\u003e contributed to data analysis, code development, final manuscript writing and provided guidance as well as financial support.\u003c/p\u003e\n\u003ch2\u003eFunding Sources\u003c/h2\u003e\n\u003cp\u003eThis research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.\u003c/p\u003e\n\u003ch2\u003eDeclaration of Competing Interest\u003c/h2\u003e\n\u003cp\u003eThe authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.\u003c/p\u003e\n\u003ch2\u003eAcknowledgements\u003c/h2\u003e\n\u003cp\u003eWe would like to thank \u003cstrong\u003eRanga Rodrigo\u003c/strong\u003e for giving valuable advice as well as learning resources regarding machine learning.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eAlfieri, L., Bisselink, B., Dottori, F., Naumann, G., de Roo, A., Salamon, P., Wyser, K., \u0026amp; Feyen, L. (2017). Global projections of river flood risk in a warmer world. \u003cem\u003eEarth\u0026rsquo;s Future\u003c/em\u003e, \u003cem\u003e5\u003c/em\u003e(2), 171\u0026ndash;182. https://doi.org/10.1002/2016EF000485\u003c/li\u003e\n\u003cli\u003eBahdanau, D., Cho, K., \u0026amp; Bengio, Y. (2014). \u003cem\u003eNeural Machine Translation by Jointly Learning to Align and Translate\u003c/em\u003e.\u003c/li\u003e\n\u003cli\u003eBoulange, J., Hanasaki, N., Yamazaki, D., \u0026amp; Pokhrel, Y. (2021). Role of dams in reducing global flood exposure under climate change. \u003cem\u003eNature Communications\u003c/em\u003e, \u003cem\u003e12\u003c/em\u003e(1), 417. https://doi.org/10.1038/s41467-020-20704-0\u003c/li\u003e\n\u003cli\u003eCastangia, M., Grajales, L. M. M., Aliberti, A., Rossi, C., Macii, A., Macii, E., \u0026amp; Patti, E. (2023). Transformer neural networks for interpretable flood forecasting. \u003cem\u003eEnvironmental Modelling and Software\u003c/em\u003e, \u003cem\u003e160\u003c/em\u003e. https://doi.org/10.1016/j.envsoft.2022.105581\u003c/li\u003e\n\u003cli\u003eCho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., \u0026amp; Bengio, Y. (2014). \u003cem\u003eLearning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation\u003c/em\u003e.\u003c/li\u003e\n\u003cli\u003e\u0026quot;Christensen, J. H., \u0026quot;Hewitson, B., \u0026quot;Busuioc, A., \u0026quot;Chen, A., \u0026quot;Gao, X., \u0026quot;Held, I., \u0026quot;Jones, R., \u0026quot;Kolli, R. K., \u0026quot;Kwon, W. T., \u0026quot;Laprise, R., \u0026quot;Magana Rueda, V., \u0026quot;Mearns, L., \u0026quot;Menendez, C. G., \u0026quot;Raisanen, J., \u0026quot;Rinke, A., \u0026quot;Sarr, A., \u0026amp; \u0026quot;Whetton, P. (2007). \u003cem\u003eRegional Climate Projections. Chapter 11\u003c/em\u003e. Cambridge University Press, Cambridge (United Kingdom). https://doi.org/https://doi.org/\u003c/li\u003e\n\u003cli\u003eChu, H., Wu, W., Wang, Q. J., Nathan, R., \u0026amp; Wei, J. (2020). An ANN-based emulation modelling framework for flood inundation modelling: Application, challenges and future directions. \u003cem\u003eEnvironmental Modelling \u0026amp; Software\u003c/em\u003e, \u003cem\u003e124\u003c/em\u003e, 104587. https://doi.org/10.1016/J.ENVSOFT.2019.104587\u003c/li\u003e\n\u003cli\u003eDe, A. C. (1997). Management of the mahaweli, a river in sri lanka. \u003cem\u003eWater International\u003c/em\u003e, \u003cem\u003e22\u003c/em\u003e(2), 98\u0026ndash;107. https://doi.org/10.1080/02508069708686678\u003c/li\u003e\n\u003cli\u003eDiyabalanage, S., Abekoon, S., Watanabe, I., Watai, C., Ono, Y., Wijesekara, S., Guruge, K. S., \u0026amp; Chandrajith, R. (2016). Has irrigated water from Mahaweli River contributed to the kidney disease of uncertain etiology in the dry zone of Sri Lanka? \u003cem\u003eEnvironmental Geochemistry and Health\u003c/em\u003e, \u003cem\u003e38\u003c/em\u003e(3), 679\u0026ndash;690. https://doi.org/10.1007/s10653-015-9749-1\u003c/li\u003e\n\u003cli\u003eDonat, M. G., Lowry, A. L., Alexander, L. V., O\u0026rsquo;Gorman, P. A., \u0026amp; Maher, N. (2016). More extreme precipitation in the world\u0026rsquo;s dry and wet regions. \u003cem\u003eNature Climate Change\u003c/em\u003e, \u003cem\u003e6\u003c/em\u003e(5), 508\u0026ndash;513. https://doi.org/10.1038/nclimate2941\u003c/li\u003e\n\u003cli\u003eDtissibe, F. Y., Ari, A. A. A., Abboubakar, H., Njoya, A. N., Mohamadou, A., \u0026amp; Thiare, O. (2024). A comparative study of Machine Learning and Deep Learning methods for flood forecasting in the Far-North region, Cameroon. \u003cem\u003eScientific African\u003c/em\u003e, \u003cem\u003e23\u003c/em\u003e, e02053. https://doi.org/10.1016/J.SCIAF.2023.E02053\u003c/li\u003e\n\u003cli\u003eElsafi, S. H. (2014). Artificial Neural Networks (ANNs) for flood forecasting at Dongola Station in the River Nile, Sudan. \u003cem\u003eAlexandria Engineering Journal\u003c/em\u003e, \u003cem\u003e53\u003c/em\u003e(3), 655\u0026ndash;662. https://doi.org/10.1016/J.AEJ.2014.06.010\u003c/li\u003e\n\u003cli\u003eFang, Z., Wang, Y., Peng, L., \u0026amp; Hong, H. (2021). Predicting flood susceptibility using LSTM neural networks. \u003cem\u003eJournal of Hydrology\u003c/em\u003e, \u003cem\u003e594\u003c/em\u003e, 125734. https://doi.org/10.1016/J.JHYDROL.2020.125734\u003c/li\u003e\n\u003cli\u003eFarsani, R. M., \u0026amp; Pazouki, E. (2021). A Transformer Self-Attention Model for Time Series Forecasting. \u003cem\u003eJournal of Electrical and Computer Engineering Innovations\u003c/em\u003e, \u003cem\u003e9\u003c/em\u003e(1), 1\u0026ndash;10. https://doi.org/10.22061/JECEI.2020.7426.391\u003c/li\u003e\n\u003cli\u003eHallegatte, S., Vogt-Schilb, A., Bangalore, M., \u0026amp; Rozenberg, J. (2017). \u003cem\u003eUnbreakable: Building the Resilience of the Poor in the Face of Natural Disasters\u003c/em\u003e. Washington, DC: World Bank. https://doi.org/10.1596/978-1-4648-1003-9\u003c/li\u003e\n\u003cli\u003eHewawasam, T. (2010). Effect of land use in the upper Mahaweli catchment area on erosion, landslides and siltation in hydropower reservoirs of Sri Lanka. \u003cem\u003eJournal of the National Science Foundation of Sri Lanka\u003c/em\u003e, \u003cem\u003e38\u003c/em\u003e(1), 3. https://doi.org/10.4038/jnsfsr.v38i1.1721\u003c/li\u003e\n\u003cli\u003eHirabayashi, Y., Mahendran, R., Koirala, S., Konoshima, L., Yamazaki, D., Watanabe, S., Kim, H., \u0026amp; Kanae, S. (2013). Global flood risk under climate change. \u003cem\u003eNature Climate Change\u003c/em\u003e, \u003cem\u003e3\u003c/em\u003e(9), 816\u0026ndash;821. https://doi.org/10.1038/nclimate1911\u003c/li\u003e\n\u003cli\u003eHochreiter, S., \u0026amp; Schmidhuber, J. (1997). Long Short-Term Memory. \u003cem\u003eNeural Computation\u003c/em\u003e, \u003cem\u003e9\u003c/em\u003e(8), 1735\u0026ndash;1780. https://doi.org/10.1162/neco.1997.9.8.1735\u003c/li\u003e\n\u003cli\u003eHyndman, R. J., \u0026amp; Khandakar, Y. (2008). \u003cem\u003eJournal of Statistical Software Automatic Time Series Forecasting: The forecast Package for R\u003c/em\u003e (Vol. 27). http://www.jstatsoft.org/\u003c/li\u003e\n\u003cli\u003eJackson, E. K., Roberts, W., Nelsen, B., Williams, G. P., Nelson, E. J., \u0026amp; Ames, D. P. (2019). Introductory overview: Error metrics for hydrologic modelling \u0026ndash; A review of common practices and an open source library to facilitate use and adoption. In \u003cem\u003eEnvironmental Modelling and Software\u003c/em\u003e (Vol. 119, pp. 32\u0026ndash;48). Elsevier Ltd. https://doi.org/10.1016/j.envsoft.2019.05.001\u003c/li\u003e\n\u003cli\u003eLee, J., Weger, R. C., Sengupta, S. K., \u0026amp; Welch, R. M. (1990). A neural network approach to cloud classification. \u003cem\u003eIEEE Transactions on Geoscience and Remote Sensing\u003c/em\u003e, \u003cem\u003e28\u003c/em\u003e(5), 846\u0026ndash;855. https://doi.org/10.1109/36.58972\u003c/li\u003e\n\u003cli\u003eLi, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y.-X., \u0026amp; Yan, X. (n.d.). \u003cem\u003eEnhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting\u003c/em\u003e.\u003c/li\u003e\n\u003cli\u003eLiu, C., Liu, D., \u0026amp; Mu, L. (2022). Improved Transformer Model for Enhanced Monthly Streamflow Predictions of the Yangtze River. \u003cem\u003eIEEE Access\u003c/em\u003e, \u003cem\u003e10\u003c/em\u003e, 58240\u0026ndash;58253. https://doi.org/10.1109/ACCESS.2022.3178521\u003c/li\u003e\n\u003cli\u003eMadhushanka, T., Jayasinghe, T., \u0026amp; Rajapakse, R. (2024). Multi Day Ahead Flood Predictionin South Asian Tropical Zone Using Deep Learning. \u003cem\u003eAvailable as a Preprint\u003c/em\u003e. https://doi.org/https://doi.org/10.21203/rs.3.rs-4070758/v1\u003c/li\u003e\n\u003cli\u003eMourato, S., Fernandez, P., Marques, F., Rocha, A., \u0026amp; Pereira, L. (2021). An interactive Web-GIS fluvial flood forecast and alert system in operation in Portugal. \u003cem\u003eInternational Journal of Disaster Risk Reduction\u003c/em\u003e, \u003cem\u003e58\u003c/em\u003e, 102201. https://doi.org/10.1016/j.ijdrr.2021.102201\u003c/li\u003e\n\u003cli\u003eNaveendrakumar, G., Vithanage, M., Kwon, H.-H., Chandrasekara, S. S. K., Iqbal, M. C. M., Pathmarajah, S., Fernando, W. C. D. K., \u0026amp; Obeysekera, J. (2019). South Asian perspective on temperature and rainfall extremes: A review. \u003cem\u003eAtmospheric Research\u003c/em\u003e, \u003cem\u003e225\u003c/em\u003e, 110\u0026ndash;120. https://doi.org/10.1016/j.atmosres.2019.03.021\u003c/li\u003e\n\u003cli\u003eParthasarathy, B., \u0026amp; Mooley, D. A. (1978). Some Features of a Long Homogeneous Series of Indian Summer Monsoon Rainfall. \u003cem\u003eMonthly Weather Review\u003c/em\u003e, \u003cem\u003e106\u003c/em\u003e(6), 771\u0026ndash;781. https://doi.org/10.1175/1520-0493(1978)106\u0026lt;0771:SFOALH\u0026gt;2.0.CO;2\u003c/li\u003e\n\u003cli\u003ePierini, N. A., Vivoni, E. R., Robles‐Morua, A., Scott, R. L., \u0026amp; Nearing, M. A. (2014). Using observations and a distributed hydrologic model to explore runoff thresholds linked with mesquite encroachment in the Sonoran Desert. \u003cem\u003eWater Resources Research\u003c/em\u003e, \u003cem\u003e50\u003c/em\u003e(10), 8191\u0026ndash;8215. https://doi.org/10.1002/2014WR015781\u003c/li\u003e\n\u003cli\u003eRahmati, O., \u0026amp; Pourghasemi, H. R. (2017). Identification of Critical Flood Prone Areas in Data-Scarce and Ungauged Regions: A Comparison of Three Data Mining Models. \u003cem\u003eWater Resources Management\u003c/em\u003e, \u003cem\u003e31\u003c/em\u003e(5), 1473\u0026ndash;1487. https://doi.org/10.1007/s11269-017-1589-6\u003c/li\u003e\n\u003cli\u003eSerinaldi, F., Loecker, F., Kilsby, C. G., \u0026amp; Bast, H. (2018). Flood propagation and duration in large river basins: a data-driven analysis for reinsurance purposes. \u003cem\u003eNatural Hazards\u003c/em\u003e, \u003cem\u003e94\u003c/em\u003e(1), 71\u0026ndash;92. https://doi.org/10.1007/s11069-018-3374-0\u003c/li\u003e\n\u003cli\u003eShelton, S., \u0026amp; Lin, Z. (2019). Streamflow variability over the Period of 1990-2014 in Mahaweli River basin, Sri Lanka and its possible mechanisms. \u003cem\u003eWater (Switzerland)\u003c/em\u003e, \u003cem\u003e11\u003c/em\u003e(12). https://doi.org/10.3390/w11122485\u003c/li\u003e\n\u003cli\u003eTamiru, H., \u0026amp; Dinka, M. O. (2021). Application of ANN and HEC-RAS model for flood inundation mapping in lower Baro Akobo River Basin, Ethiopia. \u003cem\u003eJournal of Hydrology: Regional Studies\u003c/em\u003e, \u003cem\u003e36\u003c/em\u003e, 100855. https://doi.org/10.1016/J.EJRH.2021.100855\u003c/li\u003e\n\u003cli\u003eVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., \u0026amp; Polosukhin, I. (2017). \u003cem\u003eAttention Is All You Need\u003c/em\u003e. http://arxiv.org/abs/1706.03762\u003c/li\u003e\n\u003cli\u003eWei, X., Wang, G., Schmalz, B., Hagan, D. F. T., \u0026amp; Duan, Z. (2023). Evaluate Transformer model and Self-Attention mechanism in the Yangtze River basin runoff prediction. \u003cem\u003eJournal of Hydrology: Regional Studies\u003c/em\u003e, \u003cem\u003e47\u003c/em\u003e. https://doi.org/10.1016/j.ejrh.2023.101438\u003c/li\u003e\n\u003cli\u003eWickramagamage, P. (2016). Spatial and temporal variation of rainfall trends of Sri Lanka. \u003cem\u003eTheoretical and Applied Climatology\u003c/em\u003e, \u003cem\u003e125\u003c/em\u003e(3\u0026ndash;4), 427\u0026ndash;438. https://doi.org/10.1007/s00704-015-1492-0\u003c/li\u003e\n\u003cli\u003eWu, N., Green, B., Ben, X., \u0026amp; O\u0026rsquo;Banion, S. (2020). \u003cem\u003eDeep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case\u003c/em\u003e. http://arxiv.org/abs/2001.08317\u003c/li\u003e\n\u003cli\u003eXiang, Z., Yan, J., \u0026amp; Demir, I. (2020). A Rainfall-Runoff Model With LSTM-Based Sequence-to-Sequence Learning. \u003cem\u003eWater Resources Research\u003c/em\u003e, \u003cem\u003e56\u003c/em\u003e(1). https://doi.org/10.1029/2019WR025326\u003c/li\u003e\n\u003cli\u003eXie, S.-P., \u0026amp; Saiki, N. (1999). Abrupt Onset and Slow Seasonal Evolution of Summer Monsoon in an Idealized GCM Simulation. \u003cem\u003eJournal of the Meteorological Society of Japan. Ser. II\u003c/em\u003e, \u003cem\u003e77\u003c/em\u003e(4), 949\u0026ndash;968. https://doi.org/10.2151/jmsj1965.77.4_949\u003c/li\u003e\n\u003cli\u003eXu, Y., Lin, K., Hu, C., Wang, S., Wu, Q., Zhang, L., \u0026amp; Ran, G. (2023). Deep transfer learning based on transformer for flood forecasting in data-sparse basins. \u003cem\u003eJournal of Hydrology\u003c/em\u003e, \u003cem\u003e625\u003c/em\u003e. https://doi.org/10.1016/j.jhydrol.2023.129956\u003c/li\u003e\n\u003cli\u003eZhang, Y., Ragettli, S., Molnar, P., Fink, O., \u0026amp; Peleg, N. (2022). Generalization of an Encoder-Decoder LSTM model for flood prediction in ungauged catchments. \u003cem\u003eJournal of Hydrology\u003c/em\u003e, \u003cem\u003e614\u003c/em\u003e, 128577. https://doi.org/10.1016/j.jhydrol.2022.128577\u003c/li\u003e\n\u003cli\u003eZou, Y., Wang, J., Lei, P., \u0026amp; Li, Y. (2023). A novel multi-step ahead forecasting model for flood based on time residual LSTM. \u003cem\u003eJournal of Hydrology\u003c/em\u003e, \u003cem\u003e620\u003c/em\u003e, 129521. https://doi.org/10.1016/J.JHYDROL.2023.129521\u003c/li\u003e\n\u003cli\u003eZubair, L. (2003). El Ni\u0026ntilde;o-southern oscillation influences on the Mahaweli streamflow in Sri Lanka. \u003cem\u003eInternational Journal of Climatology\u003c/em\u003e, \u003cem\u003e23\u003c/em\u003e(1), 91\u0026ndash;102. https://doi.org/10.1002/joc.865\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"University of Moratuwa","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Flood Forecasting, South Asian Tropical Zone, Mahaweli Catchment, Long Short-Term Memory, Transformer","lastPublishedDoi":"10.21203/rs.3.rs-4115691/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4115691/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThe imperative for a reliable and accurate flood forecasting procedure stem from the hazardous nature of the disaster. In response, researchers are increasingly turning to innovative approaches, particularly machine learning models, which offer enhanced accuracy compared to traditional methods. However, a notable gap exists in the literature concerning studies focused on the South Asian tropical region, which possesses distinct climate characteristics. This study investigates the applicability and behavior of Long Short-Term Memory (LSTM) and Transformer models in flood simulation with one day lead time, at the lower reach of Mahaweli catchment in Sri Lanka, which is mostly affected by the Northeast Monsoon. The importance of different input variables in the prediction was also a key focus of this study. Input features for the models included observed rainfall data collected from three nearby rain gauges, as well as historical discharge data from the target river gauge. Results showed that use of past water level data denotes a higher impact on the output compared to the other input features such as rainfall, for both architectures. All models denoted satisfactory performances in simulating daily water levels, especially low stream flows, with Nash Sutcliffe Efficiency (NSE) values greater than 0.77 while Transformer Encoder model showed a superior performance compared to Encoder Decoder models.\u003c/p\u003e","manuscriptTitle":"Behavior of LSTM and Transformer Deep Learning Models in Flood Simulation Considering South Asian Tropical Climate","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-18 12:08:37","doi":"10.21203/rs.3.rs-4115691/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"8aefe4f8-7e3b-4c70-85f6-7a79744b672a","owner":[],"postedDate":"March 18th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":29532859,"name":"Hydrology"},{"id":29532860,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2024-03-18T12:08:37+00:00","versionOfRecord":[],"versionCreatedAt":"2024-03-18 12:08:37","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4115691","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4115691","identity":"rs-4115691","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}