Multi-Modal Data Driven Algorithm for Efficient Trade Market Prediction

Multi-Modal Data Driven Algorithm for Efficient Trade Market Prediction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Multi-Modal Data Driven Algorithm for Efficient Trade Market Prediction Komal Batool, Dr. Ubaida Fatima This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6509598/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Financial market prediction is an attractive research area for the researchers as it helps the market participators to make decisions accordingly. However, the forecasting of financial market is not an easy task as the movement of financial market is stochastic in nature and is affected by several controllable and uncontrollable factors. In this research, S&P 500 index and NASDAQ is predicted using five machine learning models including support vector regression, random forest, linear regression, k nearest neighbour and LSTM. Three different datasets are used for the forecasting of daily closing price of S&P 500 index and NASDAQ in order to check the sensitivity of the market towards different factors. Firstly, historical data along with macroeconomic factors is used to design a model. Second dataset is sentiment features extracted from web news. Lastly, a hybrid data is developed by combining the first two datasets. LSTM model outperformed other machine learning models for the prediction of both financial markets. It is also observed that our developed dataset is the most efficient one as the models based on this dataset gives the minimum RMSE. Artificial Intelligence and Machine Learning Machine learning techniques Stock market predictions decision making sentiment analysis textual data Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 Background Investment in financial market is one of the most crucial decisions that a trader make. Decision making process is not an easy process. Trader is required to have an efficient understanding of market before making any decision (Gandhmal et.al, 2019 ). Movement in financial market is random or stochastic in nature. Fluctuations in financial market are affected by several internal as well as external factors (Agyarko et.al, 2023 ). Therefore, stochastic models are used to analyse the behaviour of the market so that randomness of the market can be analysed. These models incorporate the random behaviour of price movements and give better results. Financial market prediction is done by econometric models such as for forecasting of return ARMA (AutoRegressive Moving Average) (Tang et.al, 2021), ARIMA (AutoRegressive Integrated Moving Average) (Meher et.al, 2021 ) can be used. Similarly, ARCH (AutoRegressive Conditional Heteroscedasticity) (Mattera et.al, 2024), GARCH (Generalized ARCH) (Arashi et.al, 2022) can be used to predict the volatility of a market. Both risk and return are equally significant to be analysed for affective decision making (Batool et.al, 2022 ). Several machine learning algorithms like Support Vector Machine (SVM)0 (Farquad et.al, 2012 ), K-Nearest Neighbour (KNN) (Latha et.al, 2022 ), Decision Tree (Karim et.al, 2021 ), Artificial Neural Networks (ANN) (Verma et.al, 2017 ) are also used for the forecasting of future trends by either using regression or classification. Both machine learning and econometric models are stochastic in nature. There can be multiple ways to enhance the efficiency of the predictions for affective decision making. One of the approaches is by incorporating optimal features in a training of a predictive model. A predictive model can be enhanced by incorporating external economic factors. Political and financial news can also be incorporated in a predictive model using sentiment analysis (Ge, Q. et.al, 2025). News headlines and news articles are used and fed in machine learning algorithms to extract events and is then used to analyse their impact on financial market (Bhanja, S., & Das, A., 2024 ). Different textual datasets other than news can also be used to extract information to study their impact on financial market (Almeida et.al, 2021 ). This data may be collected from trends (Hu et.al, 2018 , tweets (Ni et.al, 2021 ) or any social media forums (Zhao et.al, 2020 ). In short, from historical data to macroeconomic factors and now sentiments extracted from different data resources all attributes significantly play a part in changing the prices in a stock market and hence can be incorporated in a predictive model for better forecasting (Zhong et.al, 2021). Another approach for improving the future forecasting is hybridization of a model in which multiple models are combined together for the training purpose (Chen et.al, 2025 ). As econometric model is proved to have a better short term prediction while machine learning models are good for long term prediction therefore a combine approach can be efficient for both long and short term forecasting Srivastava et.al 2022 ). Additionally, classic time series model are quite efficient to extract the linear relationship among the features involved. On the other hand, Machine learning models are used to dig out the complex and non-linear relationship among the dataset Tealab et.al, 2017 ). Method details Data Description In order to forecast daily closing price of S&P 500 index, two different types of datasets are collected from different sources. In order to identify the impact of other financial markets on S&P 500 index different technical indicators are used. Along with that to identify the predictivity of S&P 500 index based on sentiments, data from web news is collected. Data from Financial Market Data of three stock indices is collected from investing.com which includes S&P 500 index and DJIA index. Along with that from currency market daily closing price of EURUSD, GBPUSD and USD index is also collected. Daily gold price and crude oil price is also fetched from commodity market and daily bitcoin price is used from crypto currency market to understand the impact in the movement of S&P 500 index because of these indices. Daily NASDAQ index is also collected to check whether the feature variables have same impact on this index as they have on S&P 500. Figure 1 shows the five rows of dataset collected from financial markets. Data from Web News Textual data from web news is collected from two different sources. Using python libraries ‘beautiful soup’ and ‘selenium’, web scraping is done to extract news data from BBC news. Figure 2 shows the five rows collected from BBC news that contains 5 columns including heading, date, author, content and link. Similarly, for extracting news data from yahoo finance only ‘beautiful soup’ is used which extracts daily financial news. Figure 3 shows the five rows collected from yahoo finance that contains 7 columns including title, author, date, content, read time, tags and link. Data Preprocessing & Data Preparation As mentioned in a previous section that two types of datasets are collected from different data sources. First dataset is a numerical data collected from financial markets. Second dataset is a textual dataset collected from web news, which is later pre-processed to convert into numeric data by calculating sentiment polarities of news headlines and news contents. Target Variable Daily closing price of S&P 500 index Validation Variable Daily closing price of NASDAQ Features Dataset 1 Daily closing index of DJIA, bitcoin, gold price, crude oil price, exchange rate of EUR and USD, exchange rate of GBP and USD and USD index. Features dataset 2 Sentiment score calculated from daily news, average sentence length, average word length and lexical diversity of news collected on daily basis. Features Dataset 3 Hybrid dataset created by merging both datasets mentioned above. Historical Data & Technical Indicators (Dataset 1) The dataset collected from financial markets is a numeric data which includes some of the missing values that are removed by forward fill. For historical data, AIC (Akaike Information Criteria) value is obtained on Autoregressive (AR) model to identify the optimal number of previous values required to predict the future value effectively. The AIC value of AR (2) model is the least value. Therefore, previous two values are also used along with the technical indicators containing gold price, crude oil price, US Dollar index, exchange rate of GBP and USD, exchange rate of EUR and USD, closing index of NASDAQ, closing price of DJIA index and price of bitcoin. Figure 5 shows the first five rows of dataset obtained after merging two lagged values and technical indicators. Textual Data from Web News (Dataset 2) Textual data collected from web news is from two different sources and contained different attributes. Since two features daily news headlines and body of news are required, therefore all other features are omitted from the dataset. Both datasets are then merged in a single data with respect to the variable ‘Date’ which later preprocessed using the NLP preprocessing techniques as shown in Fig. 6 and Fig. 7 . Using NLTK (Natural Language Tool Kit), firstly the textual data is tokenized and then the stop words are removed. After that, PoS (Part of Speech) tagging is done. After these steps, word frequency is calculated and then syntactic features are also computed. Moreover, average value of words length and sentence length is also calculated for interpretation. Sentiment intensity analyzer is used that give the polarity of daily news of merged dataset using a python library ‘text blob’. Lexical diversity is computed by calculating the ratio of unique words and total words in a text. Figure 7 shows the developed dataset extracted from both news sources to predict the financial markets S&P 500 index and NASDAQ. All these steps of development of dataset 2 are visualized in Fig. 8 . Development of Hybrid Dataset (Dataset 3) In order to identify the optimal number of features required to predict the future behavior of S&P 500 index, above features are merged. This new dataset comprises of historical data, technical indicators and sentiment scores, lexical diversity, average word length and average sentence length. Figure 9 shows the top 5 rows of a hybrid dataset. The novelty of this dataset is its creation procedure. The complete news articles along with other details headlines, time and sources is collected on daily basis from two different portals which is further preprocessed to extract sentiment features from it. The collected dataset can be used in so many domains like sustainability and environment, health care, social and political studies and so on. Model Training After collection and preprocessing of data, the process of model deigning is started. In order to implement a model for the forecasting of future behavior of S&P 500 index, data is split into train and test. 80% of the data is used for training purpose while rest of 20% is kept for testing. Multiple statistical and machine learning models are used which include random forest, linear regression, Support Vector Regression (SVR), K-Nearest Neighbor Regression (KNR) and Long-Short Term Memory (LSTM). These models are trained on three different datasets mentioned in previous section. Figure 10 represents complete methodology from data collection to model implementation. Table 1 shows the trained model concept on considered datasets. Table 1 Trained models concept on considered dataset Name of the Model Model Concept KNN Regression KNN is an easy and non-parametric machine-learning computational metric used for classification and regression. In the context of stock market datasets, KNN can be applied to predict stock prices or classify stocks based on their performance. The algorithm/metric works by finding the "k" most similar historical data points (neighbours) to a target stock instance based on features like price, volume, or technical indicators. It then uses the average or majority label of these neighbours to predict the stock’s future price or classify it as a buy/sell signal, making KNN effective for short-term stock market predictions. Equation 1: Mathematical concept of implemented KNN \(\:\varvec{d}\left(\varvec{i},\varvec{j}\right)=\varvec{\alpha\:}.{\varvec{d}}_{\varvec{h}\varvec{i}\varvec{s}\varvec{t}\varvec{o}\varvec{r}\varvec{i}\varvec{c}\varvec{a}\varvec{l}}\left(\varvec{i},\varvec{j}\right)+\varvec{\beta\:}.{\varvec{d}}_{\varvec{t}\varvec{e}\varvec{x}\varvec{t}\varvec{u}\varvec{a}\varvec{l}}(\varvec{i},\varvec{j})\) • \(\:d\left(i,j\right)\) is the total distance between the target stock \(\:\varvec{i}\) and a neighbor stock j . • \(\:{d}_{historical}\left(i,j\right)\:\) is the distance between historical features of stock \(\:\varvec{i}\) and stock \(\:\varvec{j}\) can be obtained by utilization of Euclidean distance. • \(\:{d}_{textual}\left(i,j\right)\:\) is the distance between the sentiment scores of textual data for \(\:\varvec{i}\) and \(\:\varvec{j}\) can be calculated by the utilization of Cosine Similarity. • \(\:\varvec{\alpha\:}\) and \(\:\varvec{\beta\:}\:\) are weights for the importance of historical and textual data. Once the distances are calculated, the \(\:\varvec{k}\) closest (smallest distance) stock instances to the target stock are selected as the nearest neighbours . The KNN algorithm uses the prices of these \(\:k\:\) nearest neighbors to predict the stock price of the target instance. Typically, this is done by averaging the stock prices of the neighbours. Equation 2: Estimation of Target Variable using KNN \(\:\widehat{y}=\frac{1}{k}\sum\:_{i=1}^{k}{y}_{i}\) Thus, the distance formula is crucial in identifying which stock instances (historical and textual) should be used for predicting the target stock price. The same concept is used for other training models with different architecture. LSTM Long short-term memory is a recurrent neural network particularly used for time series analysis. LSTM model is trained for the forecasting of S&P 500 index based on three different datasets. ‘ADAM’ optimizer is adopted for training the sequential model. A model is trained on 100 epochs. Loss at each epoch is measured using mean square error. Linear Regression A linear regression model is designed to predict S&P 500 index to determine the linear relation of S&P 500 index with the features dataset. Intercept and coefficients of each feature variable is computed that represent the direction and strength of each input variable with target variable. Random forest Random forest is trained for prediction in order to handle the complex and non-linear relation of financial market with other features. After implementation of random forest, the Gini importance of each feature is calculated that represents that how much a feature significant to decrease the variance of predicted values. SVR Support Vector regression works by designing the hyperplane to fix a function in an efficient margin of tolerance. Linear kernel is used to design the SVM model for the prediction purpose of the daily closing price of S&P 500 index. Method validation and Testing All the trained models are then tested to identify the best forecasting model. In order to test the model RMSE (Root Mean Square Error) is used by changing the size of test data. In sample testing is performed along with future forecasting which include RMSE of 1 day a head prediction, 5 days ahead prediction, 15 days ahead prediction and 30 days ahead prediction to check the predictivity of the model for short term forecasting and for long term forecasting. For the validation of the model, the same models are tested on a validation dataset. NASDAQ composite is also forecasted using the designed models in order to validate the models. Findings The models are tested with the vision to analyse the best dataset required for the training of the most effeceient model. It was found that hybrid dataset which is designed by merging the historical data, economic features and sentiments extracted from news is contributing at its best for model development in order to have the short run as well as long run of S&P 500 index as shown in Fig. 11 . The validating dataset NASDAQ also supports the same result. Along with that five different machine learning models are also compared to identify the best training model including SVM, KNN, RF, LR and LSTM. LSTM among all models gives best results with minimum RMSE. Table 2 , Table 3 and Table 4 show the value of RMSE for all three datasets. Table 2 RMSE of all trained models based on dataset 1 for different forecasting periods Model Linear Regression Random Forest SVR LSTM KNN In-Sample Testing S&P 500 index 33.35998 15.7471 55.89348 0.0292 32.30884 NASDAQ 180.9118 77.1079 898.68112 0.0360 154.84245 1 day ahead forecasting S&P 500 index 32.8316 19.4719 131.70887 0.0243 22.60400 NASDAQ 301.3300 267.8722 1469.60725 0.0438 183.44800 5 days ahead forecasting S&P 500 index 42.6359 26.6976 95.78188 0.0949 24.52858 NASDAQ 198.8778 185.3781 1023.21481 0.1443 160.68738 15 days ahead forecasting S&P 500 index 30.5597 21.4583 66.80406 0.1123 34.03306 NASDAQ 201.7898 156.7328 993.90895 0.1541 146.53082 30 days ahead forecasting S&P 500 index 32.9277 26.8429 65.47770 0.3149 37.29238 NASDAQ 193.2737 161.6201 892.36488 0.1012 156.18967 Table 3 RMSE of all trained models for different forecasting period based on dataset 2 Model Linear Regression Random Forest SVR LSTM KNN In-Sample Testing S&P 500 index 220.416 86.02538 221.5584 0.1786 194.7624 NASDAQ 775.9526 301.2857 783.1157 0.1893 682.3346 1 day ahead forecasting S&P 500 index 54.35108 20.0329 19.7413 0.2927 51.5959 NASDAQ 292.3771 98.3395 148.9888 0.3222 81.2680 5 days ahead forecasting S&P 500 index 157.5122 150.1797 164.1566 0.2838 53.84908 NASDAQ 475.1392 405.9252 502.3682 0.3093 205.3482 15 days ahead forecasting S&P 500 index 184.8274 174.4644 181.9518 0.3095 159.6311 NASDAQ 620.2993 585.3715 607.9429 0.3304 560.6473 30 days ahead forecasting S&P 500 index 203.9549 212.3572 204.7145 0.3265 219.7349 NASDAQ 719.1599 749.1952 723.9399 0.3548 781.7515 Table 4 RMSE of the trained models for different forecasting periods Model Linear Regression Random Forest SVR LSTM KNN In-Sample Testing S&P 500 index 23.6028 1.78400 27.1089 0.0122 21.5062 NASDAQ 775.95262 301.28574 783.11567 0.02165 682.33463 1 day ahead forecasting S&P 500 index 44.3046 1.82E-12 44.7499 0.0251 30.63 NASDAQ 292.37709 98.33950 148.98890 0.11375 81.26800 5 days ahead forecasting S&P 500 index 28.2729 4.91E-12 28.2729 0.0232 205.1263 NASDAQ 475.13918 405.92524 475.13918 0.11977 205.34826 15 days ahead forecasting S&P 500 index 20.0122 0.18520 23.1611 0.0203 22.0902 NASDAQ 620.29928 585.37147 607.94299 0.08203 560.64736 30 days ahead forecasting S&P 500 index 18.6815 0.61598 26.5296 0.0294 34.5448 NASDAQ 719.15997 749.19518 723.93995 0.07070 781.75151 Conclusion & Future Work The objective of this study is to study the financial market behavior by integrating multiple datasets to capture the influence of both economic trends and sustainability factors on market behavior. Movement of financial market is affected by several factors which include past behavior, macroeconomic variables, political and economic conditions. In this research, the predictivity of financial market is analyzed based on different datasets. Five different predictive models are designed on three different datasets. It is found that a combined form of dataset which includes sentiments from web news, technical indicators and historical data (dataset 3) gives the most predictive model with minimum RMSE in comparison with the models based on only sentiments (dataset 2) and the model based on historical data and technical indicators (dataset 1) only. It is also observed that the maximum error is obtained when only sentiment based predictive models are designed [Figure 11 ]. Therefore, it can be concluded that S&P 500 index is not highly sensitive to news or it can also be referred as the information from external source is not completely translated to the market thus as per EMH S&P 500 index is an efficient market. The forecasting can be further improved by adding other features that affect the financial market, sentiments from different web forums or social media sites other than news portal. Further hybrid models can also be designed for the forecasting of financial market based along with hybrid dataset which may improve the predictivity of the financial market References Agyarko, K., Frempong, N. K., & Wiah, E. N. (2023). Research Article Hybrid Model for Stock Market Volatility. Almeida, M. D., Maia, V. M., Tommasetti, R., & de Oliveira Leite, R. (2021). Sentiment analysis based on a social media customised dictionary. MethodsX , 8 , 101449. Arashi, M., & Rounaghi, M. M. (2022). Analysis of market efficiency and fractal feature of NASDAQ stock exchange: Time series modeling and forecasting of stock index using ARMA-GARCH model. Future Business Journal , 8 (1), 14. Batool, K., Ahmed, M. F., & Ismail, M. A. (2022). A Hybrid Model of Machine Learning Model and Econometrics’ Model to Predict Volatility of KSE-100 Index. Reviews of Management Sciences Vol , 4 (1). Bhanja, S., & Das, A. (2024). A Black Swan event-based hybrid model for Indian stock markets’ trends prediction. Innovations in Systems and Software Engineering , 20 (2), 121-135. Chen, X., Xie, H., Li, Z., Zhang, H., Tao, X., & Wang, F. L. (2025). Sentiment analysis for stock market research: A bibliometric study. Natural Language Processing Journal , 100125. Farquad, M. A. H., Ravi, V., & Raju, S. B. (2012). Analytical CRM in banking and finance using SVM: a modified active learning-based rule extraction approach. International Journal of Electronic Customer Relationship Management , 6 (1), 48-73. Gandhmal, D. P., & Kumar, K. (2019). Systematic analysis and review of stock market prediction techniques. Computer Science Review , 34 , 100190. Ge, Q. (2025). Enhancing stock market Forecasting: A hybrid model for accurate prediction of S&P 500 and CSI 300 future prices. Expert Systems with Applications , 260 , 125380. Hu, H., Tang, L., Zhang, S., & Wang, H. (2018). Predicting the direction of stock markets using optimized neural networks with Google Trends. Neurocomputing , 285 , 188-195. Karim, R., Alam, M. K., & Hossain, M. R. (2021, August). Stock market analysis using linear regression and decision tree regression. In 2021 1st International Conference on Emerging Smart Technologies and Applications (eSmarTA) (pp. 1-6). IEEE. Latha, R. S., Sreekanth, G. R., Suganthe, R. C., Geetha, M., Selvaraj, R. E., Balaji, S., & Ponnusamy, P. P. (2022, January). Stock movement prediction using KNN machine learning algorithm. In 2022 International Conference on Computer Communication and Informatics (ICCCI) (pp. 1-5). IEEE. Mattera, R., & Otto, P. (2024). Network log-ARCH models for forecasting stock market volatility. International Journal of Forecasting . Meher, B. K., Hawaldar, I. T., Spulbar, C. M., & Birau, F. R. (2021). Forecasting stock market prices using mixed ARIMA model: A case study of Indian pharmaceutical companies. Investment Management and Financial Innovations , 18 (1), 42-54. Ni, H., Wang, S., & Cheng, P. (2021). A hybrid approach for stock trend prediction based on tweets embedding and historical prices. World Wide Web , 24 (3), 849-868. Srivastava, A. K., Srivastava, A., Singh, S., Sugandha, S., Tripta, & Gupta, S. (2022). Design of Machine-Learning Classifier for Stock Market Prediction. SN Computer Science , 3 (1), 88. Tang, H. (2021, April). Stock prices prediction based on ARMA model. In 2021 International Conference on Computer, Blockchain and Financial Development (CBFD) (pp. i-iv). IEEE. Tealab, A., Hefny, H., & Badr, A. (2017). Forecasting of nonlinear time series using ANN. Future Computing and Informatics Journal , 2 (1), 39-47 Verma, S. A., Thampi, G., & Rao, M. (2017). Inter-comparison of artificial neural network algorithms for time series forecasting: Predicting indian financial markets. International Journal of Computer Applications , 162 (2), 1-13. Zhao, J., Sun, N., & Cheng, W. (2020). Logistics forum based prediction on stock index using intelligent data analysis and processing of online web posts. Journal of Ambient Intelligence and Humanized Computing , 11 , 3575-3584. Zhong, S., & Hitchcock, D. B. (2021). S&p 500 stock price prediction using technical, fundamental and text data. arXiv preprint arXiv:2108.10826 . Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6509598","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":446725059,"identity":"f3ee6740-5d78-406e-a005-cedc0ee7ef0b","order_by":0,"name":"Komal Batool","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA4klEQVRIie3OIQvCQBTA8R2CScw3NvUrOFYEv8w9Bq4oVoPgS2dQsSqIn2EwWBYOtjL7BcOSeTaTuAkGy7k1kfvDPXjhd3eGodP9YLQc2awYp9feKE6zAmHpixCsToDXIeZyBTkcL9229JLMmA0BLc6UxGqdQwrR1TXlCNBIfUA7PilJl06CgggI5NhBwgUg9fEbCe9wEItATm9IHhWIRScRBRSsL8cECZZkpP6YuT5HAxYLZ59enR2LfZfbMVMSmmxCmc9Fr514WZ7Ph52txftK8ll5fZPWAO+H6xOdTqf7757Hc0/bVK/KDQAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0009-0001-9601-6845","institution":"NED University of Engineering \u0026 Technology","correspondingAuthor":true,"prefix":"","firstName":"Komal","middleName":"","lastName":"Batool","suffix":""},{"id":446725060,"identity":"89d8bac3-d63a-416a-a14d-cba0c5e8380f","order_by":1,"name":"Dr. Ubaida Fatima","email":"","orcid":"","institution":"NED University of Engineering \u0026 Technology","correspondingAuthor":false,"prefix":"Dr.","firstName":"Ubaida","middleName":"","lastName":"Fatima","suffix":""}],"badges":[],"createdAt":"2025-04-23 06:46:10","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-6509598/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6509598/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":81259934,"identity":"085dbcb7-927f-43f1-ab33-22cc5ba9265e","added_by":"auto","created_at":"2025-04-24 06:03:58","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":18276,"visible":true,"origin":"","legend":"\u003cp\u003eFirst 5 rows of data collected from financial markets.\u003c/p\u003e","description":"","filename":"image1.png","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/5f9f8502ae842e7cf056937e.png"},{"id":81259930,"identity":"b2cfacb8-2b0b-417c-8db8-4a7adaa47bec","added_by":"auto","created_at":"2025-04-24 06:03:58","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":32516,"visible":true,"origin":"","legend":"\u003cp\u003eSample data collected from BBC news from web scraping\u003c/p\u003e","description":"","filename":"image2.png","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/421683a26686dded501fd223.png"},{"id":81261101,"identity":"d2d5acd0-934f-4bf9-bf49-3716ff9e89f2","added_by":"auto","created_at":"2025-04-24 06:19:58","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":58417,"visible":true,"origin":"","legend":"\u003cp\u003eSample data collected from yahoo finance through web scraping\u003c/p\u003e","description":"","filename":"image3.png","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/3adfa3c76c8722f4ab284cac.png"},{"id":81259935,"identity":"b1480038-1856-41d5-a3cb-647b59281425","added_by":"auto","created_at":"2025-04-24 06:03:58","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":21456,"visible":true,"origin":"","legend":"\u003cp\u003esample datasets containing both historical data and technical indicators\u003c/p\u003e","description":"","filename":"image4.png","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/cde2e1f490aaf883ade3951e.png"},{"id":81259936,"identity":"1bc5c83f-88cd-45f8-9d2e-f84ced6ef91a","added_by":"auto","created_at":"2025-04-24 06:03:58","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":19276,"visible":true,"origin":"","legend":"\u003cp\u003eFirst 5 rows of data obtained after merging news data from BBC News and yahoo finance\u003c/p\u003e","description":"","filename":"image5.png","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/bdf2c3910d4b0826c76af2c4.png"},{"id":81261102,"identity":"28f7738a-94d2-489e-851b-cfe8577a351f","added_by":"auto","created_at":"2025-04-24 06:19:58","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":32074,"visible":true,"origin":"","legend":"\u003cp\u003eLast 5 rows of data obtained after merging news data from BBC News and yahoo finance\u003c/p\u003e","description":"","filename":"image6.png","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/d4dcab9efab41418dc5ac845.png"},{"id":81261105,"identity":"de16b758-c6fb-412b-b71c-161ecc6cae09","added_by":"auto","created_at":"2025-04-24 06:19:59","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":8834,"visible":true,"origin":"","legend":"\u003cp\u003ePrepared data from web news to design the predictive models\u003c/p\u003e","description":"","filename":"image7.png","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/096908471ddfb516e880ef07.png"},{"id":81259950,"identity":"d68eb02e-a2d2-4f7a-969e-43e12869914c","added_by":"auto","created_at":"2025-04-24 06:03:59","extension":"jpg","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":56892,"visible":true,"origin":"","legend":"\u003cp\u003eFlow of development of dataset 2\u003c/p\u003e","description":"","filename":"image8.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/c2d58431cb97c4ae3fd6906f.jpg"},{"id":81260860,"identity":"c9ceb502-6f6b-41db-9b03-9feccc333df1","added_by":"auto","created_at":"2025-04-24 06:11:58","extension":"png","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":24427,"visible":true,"origin":"","legend":"\u003cp\u003eFirst 5 rows of hybrid dataset (Dataset 3)\u003c/p\u003e","description":"","filename":"image9.png","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/c4f0c69c502e22558408e6de.png"},{"id":81259956,"identity":"6c131c7a-8539-47d5-82bd-575f8b553626","added_by":"auto","created_at":"2025-04-24 06:03:59","extension":"jpg","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":87730,"visible":true,"origin":"","legend":"\u003cp\u003eProposed Methodology\u003c/p\u003e","description":"","filename":"image10.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/4cae2e93b4e0ee328c1845ea.jpg"},{"id":81260865,"identity":"8d922230-1ebc-4de6-b8c0-5610525895b9","added_by":"auto","created_at":"2025-04-24 06:11:59","extension":"png","order_by":11,"title":"Figure 11","display":"","copyAsset":false,"role":"figure","size":14975,"visible":true,"origin":"","legend":"\u003cp\u003eRMSE of All Models on Different Datasets\u003c/p\u003e","description":"","filename":"11.png","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/5c24530b2d06dc50cf2a83b4.png"},{"id":81261694,"identity":"8867f7e8-1d3e-4874-857d-4d435c534f5f","added_by":"auto","created_at":"2025-04-24 06:28:04","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1192220,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6509598/v1/f3f742ea-5248-46fa-b06e-63840e2beff3.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eMulti-Modal Data Driven Algorithm for Efficient Trade Market Prediction\u003c/p\u003e","fulltext":[{"header":"Background","content":"\u003cp\u003eInvestment in financial market is one of the most crucial decisions that a trader make. Decision making process is not an easy process. Trader is required to have an efficient understanding of market before making any decision (Gandhmal et.al, 2019 ). Movement in financial market is random or stochastic in nature. Fluctuations in financial market are affected by several internal as well as external factors (Agyarko et.al, \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2023\u003c/span\u003e). Therefore, stochastic models are used to analyse the behaviour of the market so that randomness of the market can be analysed. These models incorporate the random behaviour of price movements and give better results. Financial market prediction is done by econometric models such as for forecasting of return ARMA (AutoRegressive Moving Average) (Tang et.al, 2021), ARIMA (AutoRegressive Integrated Moving Average) (Meher et.al, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2021\u003c/span\u003e) can be used. Similarly, ARCH (AutoRegressive Conditional Heteroscedasticity) (Mattera et.al, 2024), GARCH (Generalized ARCH) (Arashi et.al, 2022) can be used to predict the volatility of a market. Both risk and return are equally significant to be analysed for affective decision making (Batool et.al, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Several machine learning algorithms like Support Vector Machine (SVM)0 (Farquad et.al, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2012\u003c/span\u003e), K-Nearest Neighbour (KNN) (Latha et.al, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2022\u003c/span\u003e), Decision Tree (Karim et.al, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2021\u003c/span\u003e), Artificial Neural Networks (ANN) (Verma et.al, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e2017\u003c/span\u003e) are also used for the forecasting of future trends by either using regression or classification. Both machine learning and econometric models are stochastic in nature.\u003c/p\u003e \u003cp\u003eThere can be multiple ways to enhance the efficiency of the predictions for affective decision making. One of the approaches is by incorporating optimal features in a training of a predictive model. A predictive model can be enhanced by incorporating external economic factors. Political and financial news can also be incorporated in a predictive model using sentiment analysis (Ge, Q. et.al, 2025). News headlines and news articles are used and fed in machine learning algorithms to extract events and is then used to analyse their impact on financial market (Bhanja, S., \u0026amp; Das, A., \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). Different textual datasets other than news can also be used to extract information to study their impact on financial market (Almeida et.al, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). This data may be collected from trends (Hu et.al, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2018\u003c/span\u003e, tweets (Ni et.al, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2021\u003c/span\u003e) or any social media forums (Zhao et.al, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). In short, from historical data to macroeconomic factors and now sentiments extracted from different data resources all attributes significantly play a part in changing the prices in a stock market and hence can be incorporated in a predictive model for better forecasting (Zhong et.al, 2021).\u003c/p\u003e \u003cp\u003eAnother approach for improving the future forecasting is hybridization of a model in which multiple models are combined together for the training purpose (Chen et.al, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2025\u003c/span\u003e). As econometric model is proved to have a better short term prediction while machine learning models are good for long term prediction therefore a combine approach can be efficient for both long and short term forecasting Srivastava et.al \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Additionally, classic time series model are quite efficient to extract the linear relationship among the features involved. On the other hand, Machine learning models are used to dig out the complex and non-linear relationship among the dataset Tealab et.al, \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e2017\u003c/span\u003e).\u003c/p\u003e"},{"header":"Method details","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eData Description\u003c/h2\u003e \u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eIn order to forecast daily closing price of S\u0026amp;P 500 index, two different types of datasets are collected from different sources. In order to identify the impact of other financial markets on S\u0026amp;P 500 index different technical indicators are used. Along with that to identify the predictivity of S\u0026amp;P 500 index based on sentiments, data from web news is collected.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eData from Financial Market\u003c/h3\u003e\n\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eData of three stock indices is collected from investing.com which includes S\u0026amp;P 500 index and DJIA index. Along with that from currency market daily closing price of EURUSD, GBPUSD and USD index is also collected. Daily gold price and crude oil price is also fetched from commodity market and daily bitcoin price is used from crypto currency market to understand the impact in the movement of S\u0026amp;P 500 index because of these indices. Daily NASDAQ index is also collected to check whether the feature variables have same impact on this index as they have on S\u0026amp;P 500. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows the five rows of dataset collected from financial markets.\u003c/p\u003e \u003c/div\u003e \u003c/p\u003e\n\u003ch3\u003eData from Web News\u003c/h3\u003e\n\u003cp\u003eTextual data from web news is collected from two different sources. Using python libraries \u0026lsquo;beautiful soup\u0026rsquo; and \u0026lsquo;selenium\u0026rsquo;, web scraping is done to extract news data from BBC news. Figure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows the five rows collected from BBC news that contains 5 columns including heading, date, author, content and link. Similarly, for extracting news data from yahoo finance only \u0026lsquo;beautiful soup\u0026rsquo; is used which extracts daily financial news. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows the five rows collected from yahoo finance that contains 7 columns including title, author, date, content, read time, tags and link.\u003c/p\u003e\n\u003ch3\u003eData Preprocessing \u0026 Data Preparation\u003c/h3\u003e\n\u003cp\u003eAs mentioned in a previous section that two types of datasets are collected from different data sources. First dataset is a numerical data collected from financial markets. Second dataset is a textual dataset collected from web news, which is later pre-processed to convert into numeric data by calculating sentiment polarities of news headlines and news contents.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eTarget Variable\u003c/strong\u003e \u003cp\u003eDaily closing price of S\u0026amp;P 500 index\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eValidation Variable\u003c/strong\u003e \u003cp\u003eDaily closing price of NASDAQ\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eFeatures Dataset 1\u003c/strong\u003e \u003cp\u003eDaily closing index of DJIA, bitcoin, gold price, crude oil price, exchange rate of EUR and USD, exchange rate of GBP and USD and USD index.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eFeatures dataset 2\u003c/strong\u003e \u003cp\u003eSentiment score calculated from daily news, average sentence length, average word length and lexical diversity of news collected on daily basis.\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eFeatures Dataset 3\u003c/strong\u003e \u003cp\u003eHybrid dataset created by merging both datasets mentioned above.\u003c/p\u003e \u003c/p\u003e\n\u003ch3\u003eHistorical Data \u0026 Technical Indicators (Dataset 1)\u003c/h3\u003e\n\u003cp\u003eThe dataset collected from financial markets is a numeric data which includes some of the missing values that are removed by forward fill. For historical data, AIC (Akaike Information Criteria) value is obtained on Autoregressive (AR) model to identify the optimal number of previous values required to predict the future value effectively. The AIC value of AR (2) model is the least value. Therefore, previous two values are also used along with the technical indicators containing gold price, crude oil price, US Dollar index, exchange rate of GBP and USD, exchange rate of EUR and USD, closing index of NASDAQ, closing price of DJIA index and price of bitcoin. Figure\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e shows the first five rows of dataset obtained after merging two lagged values and technical indicators.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eTextual Data from Web News (Dataset 2)\u003c/h2\u003e \u003cp\u003eTextual data collected from web news is from two different sources and contained different attributes. Since two features daily news headlines and body of news are required, therefore all other features are omitted from the dataset. Both datasets are then merged in a single data with respect to the variable \u0026lsquo;Date\u0026rsquo; which later preprocessed using the NLP preprocessing techniques as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e and Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e. Using NLTK (Natural Language Tool Kit), firstly the textual data is tokenized and then the stop words are removed. After that, PoS (Part of Speech) tagging is done. After these steps, word frequency is calculated and then syntactic features are also computed. Moreover, average value of words length and sentence length is also calculated for interpretation. Sentiment intensity analyzer is used that give the polarity of daily news of merged dataset using a python library \u0026lsquo;text blob\u0026rsquo;. Lexical diversity is computed by calculating the ratio of unique words and total words in a text. Figure\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e7\u003c/span\u003e shows the developed dataset extracted from both news sources to predict the financial markets S\u0026amp;P 500 index and NASDAQ. All these steps of development of dataset 2 are visualized in Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e8\u003c/span\u003e.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eDevelopment of Hybrid Dataset (Dataset 3)\u003c/h3\u003e\n\u003cp\u003eIn order to identify the optimal number of features required to predict the future behavior of S\u0026amp;P 500 index, above features are merged. This new dataset comprises of historical data, technical indicators and sentiment scores, lexical diversity, average word length and average sentence length. Figure\u0026nbsp;\u003cspan refid=\"Fig9\" class=\"InternalRef\"\u003e9\u003c/span\u003e shows the top 5 rows of a hybrid dataset.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe novelty of this dataset is its creation procedure. The complete news articles along with other details headlines, time and sources is collected on daily basis from two different portals which is further preprocessed to extract sentiment features from it. The collected dataset can be used in so many domains like sustainability and environment, health care, social and political studies and so on.\u003c/p\u003e\n\u003ch3\u003eModel Training\u003c/h3\u003e\n\u003cp\u003eAfter collection and preprocessing of data, the process of model deigning is started. In order to implement a model for the forecasting of future behavior of S\u0026amp;P 500 index, data is split into train and test. 80% of the data is used for training purpose while rest of 20% is kept for testing. Multiple statistical and machine learning models are used which include random forest, linear regression, Support Vector Regression (SVR), K-Nearest Neighbor Regression (KNR) and Long-Short Term Memory (LSTM). These models are trained on three different datasets mentioned in previous section. Figure\u0026nbsp;\u003cspan refid=\"Fig10\" class=\"InternalRef\"\u003e10\u003c/span\u003e represents complete methodology from data collection to model implementation. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows the trained model concept on considered datasets.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eTrained models concept on considered dataset\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"2\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eName of the Model\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eModel Concept\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eKNN Regression\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eKNN is an easy and non-parametric machine-learning computational metric used for classification and regression. In the context of stock market datasets, KNN can be applied to predict stock prices or classify stocks based on their performance. The algorithm/metric works by finding the \"k\" most similar historical data points (neighbours) to a target stock instance based on features like price, volume, or technical indicators. It then uses the average or majority label of these neighbours to predict the stock\u0026rsquo;s future price or classify it as a buy/sell signal, making KNN effective for short-term stock market predictions.\u003c/em\u003e\u003c/p\u003e \u003cp\u003eEquation 1: Mathematical concept of implemented KNN\u003c/p\u003e \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varvec{d}\\left(\\varvec{i},\\varvec{j}\\right)=\\varvec{\\alpha\\:}.{\\varvec{d}}_{\\varvec{h}\\varvec{i}\\varvec{s}\\varvec{t}\\varvec{o}\\varvec{r}\\varvec{i}\\varvec{c}\\varvec{a}\\varvec{l}}\\left(\\varvec{i},\\varvec{j}\\right)+\\varvec{\\beta\\:}.{\\varvec{d}}_{\\varvec{t}\\varvec{e}\\varvec{x}\\varvec{t}\\varvec{u}\\varvec{a}\\varvec{l}}(\\varvec{i},\\varvec{j})\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003cp\u003e\u0026bull; \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:d\\left(i,j\\right)\\)\u003c/span\u003e\u003c/span\u003e \u003cem\u003eis the total distance between the target stock\u003c/em\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varvec{i}\\)\u003c/span\u003e\u003c/span\u003e \u003cem\u003eand a neighbor stock\u003c/em\u003e \u003cb\u003ej\u003c/b\u003e.\u003c/p\u003e \u003cp\u003e\u0026bull; \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{d}_{historical}\\left(i,j\\right)\\:\\)\u003c/span\u003e\u003c/span\u003e\u003cem\u003eis the distance between historical features of stock\u003c/em\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varvec{i}\\)\u003c/span\u003e\u003c/span\u003e \u003cem\u003eand stock\u003c/em\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varvec{j}\\)\u003c/span\u003e\u003c/span\u003e \u003cem\u003ecan be obtained by utilization of\u003c/em\u003e \u003cb\u003eEuclidean distance.\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u0026bull; \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{d}_{textual}\\left(i,j\\right)\\:\\)\u003c/span\u003e\u003c/span\u003e\u003cem\u003eis the distance between the sentiment scores of textual data for\u003c/em\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varvec{i}\\)\u003c/span\u003e\u003c/span\u003e \u003cem\u003eand\u003c/em\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varvec{j}\\)\u003c/span\u003e\u003c/span\u003e \u003cem\u003ecan be calculated by the utilization of\u003c/em\u003e \u003cb\u003eCosine Similarity.\u003c/b\u003e\u003c/p\u003e \u003cp\u003e\u0026bull; \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varvec{\\alpha\\:}\\)\u003c/span\u003e\u003c/span\u003e \u003cb\u003eand\u003c/b\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varvec{\\beta\\:}\\:\\)\u003c/span\u003e\u003c/span\u003e\u003cem\u003eare weights for the importance of historical and textual data.\u003c/em\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003eOnce the distances are calculated, the\u003c/em\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\varvec{k}\\)\u003c/span\u003e\u003c/span\u003e \u003cem\u003eclosest (smallest distance) stock instances to the target stock are selected as the\u003c/em\u003e \u003cb\u003enearest neighbours\u003c/b\u003e.\u003c/p\u003e \u003cp\u003e\u003cem\u003eThe KNN algorithm uses the prices of these\u003c/em\u003e \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:k\\:\\)\u003c/span\u003e\u003c/span\u003e\u003cem\u003enearest neighbors to predict the stock price of the target instance. Typically, this is done by averaging the stock prices of the neighbours.\u003c/em\u003e\u003c/p\u003e \u003cp\u003eEquation 2: Estimation of Target Variable using KNN\u003c/p\u003e \u003cp\u003e\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\widehat{y}=\\frac{1}{k}\\sum\\:_{i=1}^{k}{y}_{i}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003eThus, the\u003c/em\u003e \u003cb\u003edistance formula\u003c/b\u003e \u003cem\u003eis crucial in identifying which stock instances (historical and textual) should be used for predicting the target stock price.\u003c/em\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003eThe same concept is used for other training models with different architecture.\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLSTM\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eLong short-term memory is a recurrent neural network particularly used for time series analysis. LSTM model is trained for the forecasting of S\u0026amp;P 500 index based on three different datasets. \u0026lsquo;ADAM\u0026rsquo; optimizer is adopted for training the sequential model. A model is trained on 100 epochs. Loss at each epoch is measured using mean square error.\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eLinear Regression\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eA linear regression model is designed to predict S\u0026amp;P 500 index to determine the linear relation of S\u0026amp;P 500 index with the features dataset. Intercept and coefficients of each feature variable is computed that represent the direction and strength of each input variable with target variable.\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRandom forest\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eRandom forest is trained for prediction in order to handle the complex and non-linear relation of financial market with other features. After implementation of random forest, the Gini importance of each feature is calculated that represents that how much a feature significant to decrease the variance of predicted values.\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSVR\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cem\u003eSupport Vector regression works by designing the hyperplane to fix a function in an efficient margin of tolerance. Linear kernel is used to design the SVM model for the prediction purpose of the daily closing price of S\u0026amp;P 500 index.\u003c/em\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eMethod validation and Testing\u003c/h2\u003e \u003cp\u003eAll the trained models are then tested to identify the best forecasting model. In order to test the model RMSE (Root Mean Square Error) is used by changing the size of test data. In sample testing is performed along with future forecasting which include RMSE of 1 day a head prediction, 5 days ahead prediction, 15 days ahead prediction and 30 days ahead prediction to check the predictivity of the model for short term forecasting and for long term forecasting.\u003c/p\u003e \u003cp\u003eFor the validation of the model, the same models are tested on a validation dataset. NASDAQ composite is also forecasted using the designed models in order to validate the models.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eFindings\u003c/h2\u003e \u003cp\u003eThe models are tested with the vision to analyse the best dataset required for the training of the most effeceient model. It was found that hybrid dataset which is designed by merging the historical data, economic features and sentiments extracted from news is contributing at its best for model development in order to have the short run as well as long run of S\u0026amp;P 500 index as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig11\" class=\"InternalRef\"\u003e11\u003c/span\u003e. The validating dataset NASDAQ also supports the same result.\u003c/p\u003e \u003cp\u003eAlong with that five different machine learning models are also compared to identify the best training model including SVM, KNN, RF, LR and LSTM. LSTM among all models gives best results with minimum RMSE. Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e, Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e and Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e show the value of RMSE for all three datasets.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eRMSE of all trained models based on dataset 1 for different forecasting periods\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eModel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eLinear Regression\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eRandom Forest\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eSVR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eLSTM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eKNN\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003eIn-Sample Testing\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e33.35998\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e15.7471\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e55.89348\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.0292\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e32.30884\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e180.9118\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e77.1079\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e898.68112\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.0360\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e154.84245\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e1 day ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e32.8316\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e19.4719\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e131.70887\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.0243\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e22.60400\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e301.3300\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e267.8722\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1469.60725\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.0438\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e183.44800\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e5 days ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e42.6359\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e26.6976\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e95.78188\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.0949\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e24.52858\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e198.8778\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e185.3781\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e1023.21481\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.1443\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e160.68738\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e15 days ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e30.5597\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e21.4583\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e66.80406\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.1123\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e34.03306\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e201.7898\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e156.7328\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e993.90895\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.1541\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e146.53082\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e30 days ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e32.9277\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e26.8429\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e65.47770\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.3149\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e37.29238\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e193.2737\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e161.6201\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e892.36488\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.1012\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e156.18967\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eRMSE of all trained models for different forecasting period based on dataset 2\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eModel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eLinear Regression\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eRandom Forest\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eSVR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eLSTM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eKNN\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003eIn-Sample Testing\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e220.416\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e86.02538\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e221.5584\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.1786\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e194.7624\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e775.9526\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e301.2857\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e783.1157\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.1893\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e682.3346\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e1 day ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e54.35108\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e20.0329\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e19.7413\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.2927\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e51.5959\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e292.3771\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e98.3395\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e148.9888\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.3222\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e81.2680\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e5 days ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e157.5122\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e150.1797\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e164.1566\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.2838\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e53.84908\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e475.1392\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e405.9252\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e502.3682\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.3093\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e205.3482\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e15 days ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e184.8274\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e174.4644\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e181.9518\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.3095\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e159.6311\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e620.2993\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e585.3715\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e607.9429\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.3304\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e560.6473\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e30 days ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e203.9549\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e212.3572\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e204.7145\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.3265\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e219.7349\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e719.1599\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e749.1952\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e723.9399\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.3548\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e781.7515\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eRMSE of the trained models for different forecasting periods\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eModel\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eLinear Regression\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eRandom Forest\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eSVR\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eLSTM\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eKNN\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003eIn-Sample Testing\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e23.6028\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.78400\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e27.1089\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.0122\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e21.5062\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e775.95262\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e301.28574\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e783.11567\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.02165\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e682.33463\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e1 day ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e44.3046\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1.82E-12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e44.7499\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.0251\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e30.63\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e292.37709\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e98.33950\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e148.98890\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.11375\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e81.26800\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e5 days ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e28.2729\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.91E-12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e28.2729\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.0232\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e205.1263\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e475.13918\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e405.92524\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e475.13918\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.11977\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e205.34826\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e15 days ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e20.0122\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.18520\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e23.1611\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.0203\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e22.0902\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e620.29928\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e585.37147\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e607.94299\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.08203\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e560.64736\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003e\u003cb\u003e30 days ahead forecasting\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eS\u0026amp;P 500 index\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e18.6815\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0.61598\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e26.5296\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.0294\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e34.5448\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eNASDAQ\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e719.15997\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e749.19518\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e723.93995\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e0.07070\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e781.75151\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e "},{"header":"Conclusion \u0026 Future Work","content":"\u003cp\u003e \u003cdiv class=\"BlockQuote\"\u003e \u003cp\u003eThe objective of this study is to study the financial market behavior by integrating multiple datasets to capture the influence of both economic trends and sustainability factors on market behavior. Movement of financial market is affected by several factors which include past behavior, macroeconomic variables, political and economic conditions. In this research, the predictivity of financial market is analyzed based on different datasets. Five different predictive models are designed on three different datasets. It is found that a combined form of dataset which includes sentiments from web news, technical indicators and historical data (dataset 3) gives the most predictive model with minimum RMSE in comparison with the models based on only sentiments (dataset 2) and the model based on historical data and technical indicators (dataset 1) only. It is also observed that the maximum error is obtained when only sentiment based predictive models are designed [Figure \u003cspan refid=\"Fig11\" class=\"InternalRef\"\u003e11\u003c/span\u003e]. Therefore, it can be concluded that S\u0026amp;P 500 index is not highly sensitive to news or it can also be referred as the information from external source is not completely translated to the market thus as per EMH S\u0026amp;P 500 index is an efficient market.\u003c/p\u003e \u003cp\u003eThe forecasting can be further improved by adding other features that affect the financial market, sentiments from different web forums or social media sites other than news portal. Further hybrid models can also be designed for the forecasting of financial market based along with hybrid dataset which may improve the predictivity of the financial market\u003c/p\u003e\u003c/div\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eAgyarko, K., Frempong, N. K., \u0026amp; Wiah, E. N. (2023). Research Article Hybrid Model for Stock Market Volatility.\u003c/li\u003e\n \u003cli\u003eAlmeida, M. D., Maia, V. M., Tommasetti, R., \u0026amp; de Oliveira Leite, R. (2021). Sentiment analysis based on a social media customised dictionary. \u003cem\u003eMethodsX\u003c/em\u003e, \u003cem\u003e8\u003c/em\u003e, 101449.\u003c/li\u003e\n \u003cli\u003eArashi, M., \u0026amp; Rounaghi, M. M. (2022). Analysis of market efficiency and fractal feature of NASDAQ stock exchange: Time series modeling and forecasting of stock index using ARMA-GARCH model. \u003cem\u003eFuture Business Journal\u003c/em\u003e, \u003cem\u003e8\u003c/em\u003e(1), 14.\u003c/li\u003e\n \u003cli\u003eBatool, K., Ahmed, M. F., \u0026amp; Ismail, M. A. (2022). A Hybrid Model of Machine Learning Model and Econometrics\u0026rsquo; Model to Predict Volatility of KSE-100 Index. \u003cem\u003eReviews of Management Sciences Vol\u003c/em\u003e, \u003cem\u003e4\u003c/em\u003e(1).\u003c/li\u003e\n \u003cli\u003eBhanja, S., \u0026amp; Das, A. (2024). A Black Swan event-based hybrid model for Indian stock markets\u0026rsquo; trends prediction. \u003cem\u003eInnovations in Systems and Software Engineering\u003c/em\u003e, \u003cem\u003e20\u003c/em\u003e(2), 121-135.\u003c/li\u003e\n \u003cli\u003eChen, X., Xie, H., Li, Z., Zhang, H., Tao, X., \u0026amp; Wang, F. L. (2025). Sentiment analysis for stock market research: A bibliometric study. \u003cem\u003eNatural Language Processing Journal\u003c/em\u003e, 100125.\u003c/li\u003e\n \u003cli\u003eFarquad, M. A. H., Ravi, V., \u0026amp; Raju, S. B. (2012). Analytical CRM in banking and finance using SVM: a modified active learning-based rule extraction approach. \u003cem\u003eInternational Journal of Electronic Customer Relationship Management\u003c/em\u003e, \u003cem\u003e6\u003c/em\u003e(1), 48-73.\u003c/li\u003e\n \u003cli\u003eGandhmal, D. P., \u0026amp; Kumar, K. (2019). Systematic analysis and review of stock market prediction techniques. \u003cem\u003eComputer Science Review\u003c/em\u003e, \u003cem\u003e34\u003c/em\u003e, 100190.\u003c/li\u003e\n \u003cli\u003eGe, Q. (2025). Enhancing stock market Forecasting: A hybrid model for accurate prediction of S\u0026amp;P 500 and CSI 300 future prices. \u003cem\u003eExpert Systems with Applications\u003c/em\u003e, \u003cem\u003e260\u003c/em\u003e, 125380.\u003c/li\u003e\n \u003cli\u003eHu, H., Tang, L., Zhang, S., \u0026amp; Wang, H. (2018). Predicting the direction of stock markets using optimized neural networks with Google Trends. \u003cem\u003eNeurocomputing\u003c/em\u003e, \u003cem\u003e285\u003c/em\u003e, 188-195.\u003c/li\u003e\n \u003cli\u003eKarim, R., Alam, M. K., \u0026amp; Hossain, M. R. (2021, August). Stock market analysis using linear regression and decision tree regression. In \u003cem\u003e2021 1st International Conference on Emerging Smart Technologies and Applications (eSmarTA)\u003c/em\u003e (pp. 1-6). IEEE.\u003c/li\u003e\n \u003cli\u003eLatha, R. S., Sreekanth, G. R., Suganthe, R. C., Geetha, M., Selvaraj, R. E., Balaji, S., \u0026amp; Ponnusamy, P. P. (2022, January). Stock movement prediction using KNN machine learning algorithm. In \u003cem\u003e2022 International Conference on Computer Communication and Informatics (ICCCI)\u003c/em\u003e (pp. 1-5). IEEE.\u003c/li\u003e\n \u003cli\u003eMattera, R., \u0026amp; Otto, P. (2024). Network log-ARCH models for forecasting stock market volatility. \u003cem\u003eInternational Journal of Forecasting\u003c/em\u003e.\u003c/li\u003e\n \u003cli\u003eMeher, B. K., Hawaldar, I. T., Spulbar, C. M., \u0026amp; Birau, F. R. (2021). Forecasting stock market prices using mixed ARIMA model: A case study of Indian pharmaceutical companies. \u003cem\u003eInvestment Management and Financial Innovations\u003c/em\u003e, \u003cem\u003e18\u003c/em\u003e(1), 42-54.\u003c/li\u003e\n \u003cli\u003eNi, H., Wang, S., \u0026amp; Cheng, P. (2021). A hybrid approach for stock trend prediction based on tweets embedding and historical prices. \u003cem\u003eWorld Wide Web\u003c/em\u003e, \u003cem\u003e24\u003c/em\u003e(3), 849-868.\u003c/li\u003e\n \u003cli\u003eSrivastava, A. K., Srivastava, A., Singh, S., Sugandha, S., Tripta, \u0026amp; Gupta, S. (2022). Design of Machine-Learning Classifier for Stock Market Prediction. \u003cem\u003eSN Computer Science\u003c/em\u003e, \u003cem\u003e3\u003c/em\u003e(1), 88.\u003c/li\u003e\n \u003cli\u003eTang, H. (2021, April). Stock prices prediction based on ARMA model. In \u003cem\u003e2021 International Conference on Computer, Blockchain and Financial Development (CBFD)\u003c/em\u003e (pp. i-iv). IEEE.\u003c/li\u003e\n \u003cli\u003eTealab, A., Hefny, H., \u0026amp; Badr, A. (2017). Forecasting of nonlinear time series using ANN. \u003cem\u003eFuture Computing and Informatics Journal\u003c/em\u003e, \u003cem\u003e2\u003c/em\u003e(1), 39-47\u003c/li\u003e\n \u003cli\u003eVerma, S. A., Thampi, G., \u0026amp; Rao, M. (2017). Inter-comparison of artificial neural network algorithms for time series forecasting: Predicting indian financial markets. \u003cem\u003eInternational Journal of Computer Applications\u003c/em\u003e, \u003cem\u003e162\u003c/em\u003e(2), 1-13.\u003c/li\u003e\n \u003cli\u003eZhao, J., Sun, N., \u0026amp; Cheng, W. (2020). Logistics forum based prediction on stock index using intelligent data analysis and processing of online web posts. \u003cem\u003eJournal of Ambient Intelligence and Humanized Computing\u003c/em\u003e, \u003cem\u003e11\u003c/em\u003e, 3575-3584.\u003c/li\u003e\n \u003cli\u003eZhong, S., \u0026amp; Hitchcock, D. B. (2021). S\u0026amp;p 500 stock price prediction using technical, fundamental and text data. \u003cem\u003earXiv preprint arXiv:2108.10826\u003c/em\u003e.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Machine learning techniques, Stock market predictions, decision making, sentiment analysis, textual data","lastPublishedDoi":"10.21203/rs.3.rs-6509598/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6509598/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eFinancial market prediction is an attractive research area for the researchers as it helps the market participators to make decisions accordingly. However, the forecasting of financial market is not an easy task as the movement of financial market is stochastic in nature and is affected by several controllable and uncontrollable factors. In this research, S\u0026amp;P 500 index and NASDAQ is predicted using five machine learning models including support vector regression, random forest, linear regression, k nearest neighbour and LSTM. Three different datasets are used for the forecasting of daily closing price of S\u0026amp;P 500 index and NASDAQ in order to check the sensitivity of the market towards different factors. Firstly, historical data along with macroeconomic factors is used to design a model. Second dataset is sentiment features extracted from web news. Lastly, a hybrid data is developed by combining the first two datasets. LSTM model outperformed other machine learning models for the prediction of both financial markets. It is also observed that our developed dataset is the most efficient one as the models based on this dataset gives the minimum RMSE.\u003c/p\u003e","manuscriptTitle":"Multi-Modal Data Driven Algorithm for Efficient Trade Market Prediction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-24 06:03:54","doi":"10.21203/rs.3.rs-6509598/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2d817d74-0bcb-453d-a363-34cf87aa623a","owner":[],"postedDate":"April 24th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":47548799,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2025-04-24T06:03:54+00:00","versionOfRecord":[],"versionCreatedAt":"2025-04-24 06:03:54","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6509598","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6509598","identity":"rs-6509598","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}