Evaluation of Sentiment Analysis in Finance: From Lexicons to Transformers

Abstract

Financial and economic news is continuously monitored by financial market participants. According to the efficient market hypothesis, all past information is reflected in stock prices and new information is instantaneously absorbed in determining future stock prices. Hence, prompt extraction of positive or negative sentiments from news is very important for investment decision-making by traders, portfolio managers and investors. Sentiment analysis models can provide an efficient method for extracting actionable signals from the news. However, financial sentiment analysis is challenging due to domainspecific language and unavailability of large labeled datasets. General sentiment analysis models are ineffective when applied to specific domains such as finance. To overcome these challenges, we design an evaluation platform which we use to assess the effectiveness and performance of various sentiment analysis approaches, based on combinations of text representation methods and machine-learning classifiers. We perform more than one hundred experiments using publicly available datasets, labeled by financial experts. We start the evaluation with specific lexicons for sentiment analysis in finance and gradually build the study to include word and sentence encoders, up to the latest available NLP transformers. The results show improved efficiency of contextual embeddings in sentiment analysis compared to lexicons and fixed word and sentence encoders, even when large datasets are not available. Furthermore, distilled versions of NLP transformers produce comparable results to their larger teacher models, which makes them suitable for use in production environments.

Research on Systemic Financial Risk Measurement Based on HMM and Text Mining: A Case of China Financial Market

“…Financial and economic news is continuously monitored by financial market participants [25]. The keyword search volume reflected the activity and prosperity of the economic market.…”

Deep Neural Network for Gender-Based Violence Detection on Twitter Messages

“…Sentiment analysis has been positioned as one of the essential tools to transform the emotions and attitudes of a text into actionable and understandable information for a machine [44]. It is so important within the NLP that this area has been addressed at 3 different levels [42]: (1) the document level, focused on determining whether an opinion document expresses a positive or negative sentiment, (2) the sentence level, whose task is to check whether each sentence expresses a positive, negative or neutral opinion and (3) the aspect level, responsible for looking directly at the opinion itself.…”

Analysis of Cryptocurrency Interdependencies

“…To compute the sentiment of the news related to cryptocurrencies, we utilize the sentiment analysis model [20] based on RoBERTa transformer architecture [13], and fine-tune it on a dataset of short financial news. The model has an accuracy of 94.1% when used to predict the sentiment of short finance news [20].…”

“…The third approach forms a network of cryptocurrencies, based on the relationships between their sentiment, extracted from Reddit titles. This approach utilizes the sentiment evaluation model [20] to calculate the cumulative daily positive sentiment for each cryptocurrency and then use it to build the network. (4) Co-occurrences of cryptocurrencies in Google news.…”