Performance Evaluation of Word and Sentence Embeddings for Finance Headlines Sentiment Analysis

Mishev, Kostadin; Gjorgjevikj, Ana; Stojanov, Riste; Mishkovski, Igor; Vodenska, Irena; Chitkushev, Ljubomir; Trajanov, Dimitar

doi:10.1007/978-3-030-33110-8_14

Cited by 11 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al [ 19 ] utilized CNN to extract features and then calculated the attention weight of extracted features. Mishev et al [ 20 , 21 , 22 ] adopted Bi-sequence Gated Recurrent Unit (Bi-GRU) to extract the emotional features in the text and evaluated the model via the attention mechanism. Yang et al [ 23 ] proposed a Hierarchical Attention Network (HAN), which applies different attention mechanisms to words and sentences in the text from two dimensions, respectively.…”

Section: Related Workmentioning

confidence: 99%

Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU

Zhang

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

In the task of text sentiment analysis, the main problem that we face is that the traditional word vectors represent lack of polysemy, the Recurrent Neural Network cannot be trained in parallel, and the classification accuracy is not high. We propose a sentiment classification model based on the proposed Sliced Bidirectional Gated Recurrent Unit (Sliced Bi-GRU), Multi-head Self-Attention mechanism, and Bidirectional Encoder Representations from Transformers embedding. First, the word vector representation obtained by the BERT pre-trained language model is used as the embedding layer of the neural network. Then the input sequence is sliced into subsequences of equal length. And the Bi-sequence Gated Recurrent Unit is applied to extract the subsequent feature information. The relationship between words is learned sequentially via the Multi-head Self-attention mechanism. Finally, the emotional tendency of the text is output by the Softmax function. Experiments show that the classification accuracy of this model on the Yelp2015 dataset and the Amazon dataset is 74.37% and 62.57%, respectively. And the training speed of the model is better than most existing models, which verifies the effectiveness of the model.

show abstract

Section: Related Workmentioning

confidence: 99%

Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU

Zhang

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…We then add an attention layer after the hidden layer to evaluate its effectiveness. Furthermore, we build an additional group of GRU and LSTM networks, which support bidirectional feature extraction, to assess their performance in finance-based sentiment analysis as described in [84]. and we use binary cross-entropy loss function when training the models.…”

Section: B Deep-neural Network (Dnn)mentioning

confidence: 99%

Evaluation of Sentiment Analysis in Finance: From Lexicons to Transformers

et al. 2020

Self Cite

View full text Add to dashboard Cite

Financial and economic news is continuously monitored by financial market participants. According to the efficient market hypothesis, all past information is reflected in stock prices and new information is instantaneously absorbed in determining future stock prices. Hence, prompt extraction of positive or negative sentiments from news is very important for investment decision-making by traders, portfolio managers and investors. Sentiment analysis models can provide an efficient method for extracting actionable signals from the news. However, financial sentiment analysis is challenging due to domainspecific language and unavailability of large labeled datasets. General sentiment analysis models are ineffective when applied to specific domains such as finance. To overcome these challenges, we design an evaluation platform which we use to assess the effectiveness and performance of various sentiment analysis approaches, based on combinations of text representation methods and machine-learning classifiers. We perform more than one hundred experiments using publicly available datasets, labeled by financial experts. We start the evaluation with specific lexicons for sentiment analysis in finance and gradually build the study to include word and sentence encoders, up to the latest available NLP transformers. The results show improved efficiency of contextual embeddings in sentiment analysis compared to lexicons and fixed word and sentence encoders, even when large datasets are not available. Furthermore, distilled versions of NLP transformers produce comparable results to their larger teacher models, which makes them suitable for use in production environments.

show abstract

“…Machine learning has been used to extract finance sentiments [18] to study stock price movements [6,24], the effect of news sentiments on corporate performance [19] and financial markets [4], and to forecast Bitcoin prices [5].…”

Section: Literature Reviewmentioning

confidence: 99%