Using dynamic semantic structure of news flow to enhance financial forecasting: a twelve-year study on twitter news channels

Abstract

This research holds significance for advancing financial forecasting methodologies by shifting the focus from traditional sentiment analysis of individual tweets to exploring intricate semantic relationships within news tweets from top-followed news channels on Twitter. Addressing a notable research gap in financial forecasting, often dominated by sentiment analysis, our study endeavors to fill the void left by the underexplored intricate relationships within news entities and their dynamic semantic evolution. Motivated by the inherent challenges in predicting the random walk behavior of stock prices, we contend that incorporating longitudinal data derived from the semantic relationships between news entities can enhance the accuracy of stock market forecasts. The study pioneers a twelve-year exploration, encompassing data from 55 leading news channels on Twitter, boasting a collective following of 714 million users. The approach employs natural language processing (NLP) to extract two million unique entities, whose semantics are analyzed through complex network analysis, laying the foundation for the forecasting model. Finally, this research introduces a model linked to the dynamic semantic structure of news flow. The predictive model considers the impact of exogenous variables influenced by the evolving relationships among news entities. The results offer a proof of concept, highlighting the potential of utilizing dynamic semantic relationships among news entities for financial prediction. On average, the model demonstrates an improvement in accuracy of 40.3% across ten different stock price predictions. These findings are expounded through relevant theories, offering a theoretical foundation for observed patterns and indicating a promising direction for future research in this domain.