Stock index trend prediction based on TabNet feature selection and long short-term memory

Abstract

In this study, we propose a predictive model TabLSTM that combines machine learning methods such as TabNet and Long Short-Term Memory Neural Network (LSTM) with a complete factor library for stock index trend prediction. Our motivation is based on the notion that there are numerous interrelated factors in the stock market, and the factors that affect each stock are different. Therefore, a complete factor library and an efficient feature selection technique are necessary to predict stock index. In this paper, we first build a factor database that includes macro, micro and technical indicators. Successively, we calculate the factor importance through TabNet and rank them. Based on a prespecified threshold, the optimal factors set will include only the highest-ranked factors. Finally, using the optimal factors set as input information, LSTM is employed to predict the future trend of 4 stock indices. Empirical validation of the model shows that the combination of TabNet for factors selection and LSTM outperforms existing methods. Moreover, constructing a factor database is necessary for stock index prediction. The application of our method does not only show the feasibility to predict stock indices across different financial markets, yet it also provides an complete factor database and a comprehensive architecture for stock index trend prediction, which may provide some references for stock forecasting and quantitative investments.

An Ensemble Learning Approach Based on TabNet and Machine Learning Models for Cheating Detection in Educational Tests

“…The architecture of TabNet, as shown in Figure 1, synergistically integrates the advantages of deep learning and decision trees, enabling efficient handling of tabular data by seamlessly incorporating feature selection and prediction within a single framework. Extensive empirical evaluations demonstrate TabNet’s ability to achieve outstanding performance across diverse tabular data sets, consequently fostering its widespread adoption among machine learning practitioners engaged in structured data analysis (Borghini & Giannetti, 2021; Joseph et al, 2022; McDonnell et al, 2023; Wei et al, 2022; Yan et al, 2021).…”