Pretraining and Fine-Tuning Strategies for Sentiment Analysis of Latvian Tweets

Thakkar, Gaurish; Pinnis, Mārcis

doi:10.3233/faia200602

Cited by 4 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We adapted the sentiment analysis tool 1 used for general Latvian tweets [11] and used it along with a Latvian-specific language model [13], which we fine-tuned onto the entire LTEC. This allowed us to train a sentiment classifier to distinguish negative, positive and neutral tweets in the sentiment training data subset of LTEC other Latvian tweet data sets 2 [7,8,12].…”

Section: Methodsmentioning

confidence: 99%

Fragmented and Valuable: Following Sentiment Changes in Food Tweets

Kale¹,

Rikters²

2021

Preprint

View full text Add to dashboard Cite

We analysed sentiment and frequencies related to smell, taste and temperature expressed by food tweets in the Latvian language. To get a better understanding of the role of smell, taste and temperature in the mental map of food associations, we looked at such categories as 'tasty' and 'healthy', which turned out to be mutually exclusive. By analysing the occurrence frequency of words associated with these categories, we discovered that food discourse overall was permeated by 'tasty' while the category of 'healthy' was relatively small. Finally, we used the analysis of temporal dynamics to see if we can trace seasonality or other temporal aspects in smell, taste and temperature as reflected in food tweets. Understanding the composition of social media content with relation to smell, taste and temperature in food tweets allows us to develop our work further -on food culture/seasonality and its relation to temperature, on our limited capacity to express smell-related sentiments, and the lack of the paradigm of taste in discussing food healthiness.CCS Concepts: • Computing methodologies → Natural language processing.

show abstract

Section: Methodsmentioning

confidence: 99%

Fragmented and Valuable: Following Sentiment Changes in Food Tweets

Kale¹,

Rikters²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The most recent works proposed language models specifically pre-trained on tweet corpora: Thakkar and Pinnis [ 16 ] achieved encouraging performance leveraging a time-balanced evaluation set for sentiment analysis on Latvian tweets, comparing several BERT-based architectures, and Nguyen et al [ 12 ] presented BERTweet, the first public large-scale pre-trained language model for English tweets; Ángel González et al [ 15 ] proposed TWiLBERT, a specialization of the BERT architecture both for the Spanish language and the Twitter domain. For languages other than English, such as Persian [ 53 ] and Arabic [ 54 ], recent studies have also focused on deep neural networks such as CNN and LSTM.…”

Section: Background and Related Workmentioning

confidence: 99%

“…In the field of sentiment analysis of tweets, most of the scientific literature has obtained state-of-the-art results adopting the approach of training language models directly from scratch starting from corpora made up exclusively of tweets, so that the models could better handle the specific tweet jargon, characterized by a particular syntax and grammar not containing punctuation, with contracted or elongated words, keywords, hashtags, emoticons, emojis and so on. These approaches, working not only in English [ 11 , 12 ], but also in other languages such as Italian [ 13 ], Spanish [ 14 , 15 ], and Latvian [ 16 ], necessarily impose two constraints: the first requires the building of large corpora of tweets to be used for training the language models in the specific language considered, and the second is the need for substantial resources, of both hardware and time, to train the models from scratch starting from these corpora.…”

Section: Introductionmentioning

confidence: 99%

An Effective BERT-Based Pipeline for Twitter Sentiment Analysis: A Case Study in Italian

Pota

Ventura

Catelli

et al. 2020

Sensors

104

View full text Add to dashboard Cite

Over the last decade industrial and academic communities have increased their focus on sentiment analysis techniques, especially applied to tweets. State-of-the-art results have been recently achieved using language models trained from scratch on corpora made up exclusively of tweets, in order to better handle the Twitter jargon. This work aims to introduce a different approach for Twitter sentiment analysis based on two steps. Firstly, the tweet jargon, including emojis and emoticons, is transformed into plain text, exploiting procedures that are language-independent or easily applicable to different languages. Secondly, the resulting tweets are classified using the language model BERT, but pre-trained on plain text, instead of tweets, for two reasons: (1) pre-trained models on plain text are easily available in many languages, avoiding resource- and time-consuming model training directly on tweets from scratch; (2) available plain text corpora are larger than tweet-only ones, therefore allowing better performance. A case study describing the application of the approach to Italian is presented, with a comparison with other Italian existing solutions. The results obtained show the effectiveness of the approach and indicate that, thanks to its general basis from a methodological perspective, it can also be promising for other languages.

show abstract

“…They used the Italian language as a case study but generally, their approach is applicable to different languages. Several papers have focused on pre-training on specific-language tweet corpora: Thakkar et al [16] presented different pre-training strategies for the sentiment classification task in Latvian; Jose Angel et al [17] proposed TWilBert, a BERT-based architecture that outperforms multilingual BERT on classification tasks; and Tuan Anh et al [18] used CNN and LSTM architecture for sentiment analysis of informal Indonesian tweets. As a summary of the limitations and difficulties of sentiment analysis, a multi-class classification approach for Twitter is proposed in [19].…”

Section: Introductionmentioning

confidence: 99%

Language-Agnostic Method for Sentiment Analysis of Twitter

Jafari,

Farahbakhsh,

Salehi

et al. 2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

With the different events and crises that we are witnessing these days, Twitter plays an essential role in sharing thoughts, opinions, and news worldwide in various languages. Understanding the sentiment of user-generated content has garnered much interest in both industrial and academic communities in recent studies. Due to the limited availability of data from low-resource languages, the focus on multilingual resources is a limiting and challenging issue of sentiment analysis task. Considering the importance of pre-processing in the implementation of a sentiment analysis system, we propose a method consisting of two steps for the pre-processing of tweets in different languages i) a language-agnostic step to replace or remove some elements in the Twitter data structure and ii) a text-normalization step based on the main high-resource language. In addition, we used machine translation techniques to translate low-resource language texts into the main language. We evaluated sentiment classification approaches based on four deep models: an RNN model and three BERT-based architectures namely vanilla-version, a languagespecific and a large-scale pre-trained model for Twitter. The results show that our method had better accuracy when using a large-scale BERT-based pre-trained model.

show abstract

Pretraining and Fine-Tuning Strategies for Sentiment Analysis of Latvian Tweets

Cited by 4 publications

References 15 publications

Fragmented and Valuable: Following Sentiment Changes in Food Tweets

Fragmented and Valuable: Following Sentiment Changes in Food Tweets

An Effective BERT-Based Pipeline for Twitter Sentiment Analysis: A Case Study in Italian

Language-Agnostic Method for Sentiment Analysis of Twitter

Contact Info

Product

Resources

About