Integrating Social Media Data and Historical Stock Prices for Predictive Analysis: A Reinforcement Learning Approach

Abstract

The reliance on data collection for assessing individual behavior and actions has intensified, particularly with the proliferation of digital platforms. People often use the Internet to express their opinions and experiences about various products and services on social media and personal websites. Concurrently, the stock market, a key driver of commercial and industrial growth, has seen a surge in research focused on predicting market trends. The vast array of information on social media regarding public sentiment towards current events, coupled with the known impact of financial news on stock prices, has led to the application of data mining techniques for understanding market volatility. This research proposes a novel method that integrates social media data, encompassing public sentiment, news, and historical stock prices, to predict future stock trends. The approach involves two primary phases. The first phase develops a sentiment analysis (SA) model using three dilated convolution layers for feature extraction and classification. Addressing the challenge of unbalanced classification, a reinforcement learning (RL)-based strategy is employed, wherein an agent receives varied rewards for accurate classification, with a bias towards the minority class. Additionally, a unique clustering-based mutation operator within a differential equation (DE) framework is introduced to initiate the backpropagation (BP) process. The second phase incorporates an attention-based long short-term memory (LSTM) model, merging historical stock prices with sentiment data. An experimental analysis of the study dataset is conducted to determine optimal values for significant parameters, including the reward function.