Multiple Objectives Escaping Bird Search Optimization and Its application in Stock Market Prediction Based on Transformer Model

Abstract Stock market prediction is a popular topic in both academia and industry due to its potential to offer significant financial returns, while transformer model is a cutting-edge tool in this task. However, how to fine-tune the hyperparameters in a reasonable timeframe without excessive computational resources remains a challenge. This paper purposes an innovative algorithm which extends the Escaping Bird Search optimization (EBS) into the area of multi-objective optimization, namely MOEBS, for efficient and effective hyperparameter fine-tuning of Transformer to predict stock prices. Initially, we validate MOEBS by conducting benchmark testing built upon widely used problem sets like ZDT, DTLZ, WFG, etc., together with numerical experiments based on evaluation of 4 recognized metrics, GD, Spacing, IGD, and HV. Next, we apply MOEBS to optimal hyperparameter searching for Transformer, such as Learning rate, Number of Heads and L2 regularization coefficient. The fine-tuned Transformer models by MOEBS and other competing algorithms are then applied to training and predicting on Google's historical stock price data set from 2016 to 2021. Comparative experiments demonstrate the extraordinary excellence of MOEBS-Transformer in terms of RMSE, RPD, and R2 metrics, matching and surpassing the performance of the state-of-the-art competitors. As a novel and powerful stock price prediction model, MOEBS-Transformer has potential to become a reliable and accurate predictor for stock prices. The fine-tuned Transformer models by MOEBS and other competing algorithms are then applied to training and predicting on Google's historical stock price data set from 2016 to 2021. The Google stock data set used in this research can be accessed publicly through Kaggle at the following link: https://www.kaggle.com/datasets/varpit94/google-stock-data.