Linear Transformations for Cross-lingual Sentiment Analysis

Abstract

This paper deals with cross-lingual sentiment analysis in Czech, English and French languages. We perform zero-shot cross-lingual classification using five linear transformations combined with LSTM and CNN based classifiers. We compare the performance of the individual transformations, and in addition, we confront the transformation-based approach with existing state-of-the-art BERT-like models. We show that the pre-trained embeddings from the target domain are crucial to improving the cross-lingual classification results, unlike in the monolingual classification, where the effect is not so distinctive.

Systematic Evaluation of Linear Transformations for Cross-lingual Semantic Spaces

“…Přibáň and Steinberger (2021) investigates the usage of multilingual and monolingual transformer-based models for polarity detection in the Czech language, showing that multilingual models can achieve new state-of-the-art results and transfer knowledge from English to Czech. A subsequent work demonstrates the effectiveness of pre-trained embeddings and linear transformations for cross-lingual sentiment analysis, using LSTM and CNN classifiers, and compares their approach with existing BERT-like models (Přibáň et al, 2022).…”

“…The IMDB dataset (Maas et al, 2011) for English reviews contains 50 thousand movie reviews with positive and negative classes. Both datasets were used for sentiment analysis experiments in (Přibáň et al, 2022). As in the case of document classification, the F-measure was used to evaluate the final performance of this experiment.…”

“…The bottom of the Table 12 shows the results for a similar experiment presented in (Přibáň et al, 2022). The difference with our article is that the authors pre-trained the semantic spaces on given datasets to further improve the performance of the tested methods.…”

“…In this paper, we focus on linear transformations, which, compared to complex transformer architectures (Vaswani et al, 2017), have significantly fewer parameters (they use only a single matrix), are computationally more efficient, and can achieve competitive results, as shown in Přibáň et al (2022).…”