Learning Implicit Sentiment in Aspect-based Sentiment Analysis with Supervised Contrastive Pre-Training

Li, Zhengyan; Zou, Yicheng; Zhang, Chong; Zhang, Qi; Wei, Zhongyu

doi:10.18653/v1/2021.emnlp-main.22

Cited by 63 publications

(27 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The transcripts used for the AD diagnosis of spontaneous speech contain no polarity markers; however, most previous studies in this field generally pay little attention to implicit sentiment expressions. The study ( 44 ) used supervised contrastive learning to capture implicit sentiment using an advanced method. That is, the expressions with the same sentiment polarity were pulled together, and those with different sentiment orientations were pushed apart.…”

Section: Future Workmentioning

confidence: 99%

A Transfer Learning Method for Detecting Alzheimer's Disease Based on Speech and Natural Language Processing

Liu

Luo

Yuan

et al. 2022

Front. Public Health

View full text Add to dashboard Cite

Alzheimer's disease (AD) is a neurodegenerative disease that is difficult to be detected using convenient and reliable methods. The language change in patients with AD is an important signal of their cognitive status, which potentially helps in early diagnosis. In this study, we developed a transfer learning model based on speech and natural language processing (NLP) technology for the early diagnosis of AD. The lack of large datasets limits the use of complex neural network models without feature engineering, while transfer learning can effectively solve this problem. The transfer learning model is firstly pre-trained on large text datasets to get the pre-trained language model, and then, based on such a model, an AD classification model is performed on small training sets. Concretely, a distilled bidirectional encoder representation (distilBert) embedding, combined with a logistic regression classifier, is used to distinguish AD from normal controls. The model experiment was evaluated on Alzheimer's dementia recognition through spontaneous speech datasets in 2020, including the balanced 78 healthy controls (HC) and 78 patients with AD. The accuracy of the proposed model is 0.88, which is almost equivalent to the champion score in the challenge and a considerable improvement over the baseline of 75% established by organizers of the challenge. As a result, the transfer learning method in this study improves AD prediction, which does not only reduces the need for feature engineering but also addresses the lack of sufficiently large datasets.

show abstract

Section: Future Workmentioning

confidence: 99%

A Transfer Learning Method for Detecting Alzheimer's Disease Based on Speech and Natural Language Processing

Liu

Luo

Yuan

et al. 2022

Front. Public Health

View full text Add to dashboard Cite

show abstract

“…In recent years, a large number of outstanding opensource ASC (Li et al, 2021a;Tian et al, 2021;Li et al, 2021b; and ATESC (Li et al, 2018b;Xu et al, 2018;Ma et al, 2019;Yang, 2019;…”

Section: Related Workmentioning

confidence: 99%

PyABSA: A Modularized Framework for Reproducible Aspect-based Sentiment Analysis

Heng¹,

Li²

2022

Preprint

View full text Add to dashboard Cite

Aspect-based sentiment analysis (ABSA) has become a prevalent task in recent years. However, the absence of a unified framework in the present ABSA research makes it challenging to compare different models' performance fairly. Therefore, we develop an opensource ABSA framework, namely PYABSA. Besides, previous efforts usually neglect the precursor aspect term extraction (ASC) subtask and focus on the aspect sentiment classification (ATE) subtask, while PYABSA includes the features of aspect term extraction, aspect sentiment classification, and text classification. Furthermore, multiple ABSA subtasks can be adapted to PYABSA owing to its modular architecture. To facilitate ABSA applications, PYABSAseamless integrates multilingual modelling, automated dataset annotation, etc., which are helpful in deploying ABSA services. In ASC and ATE, PYABSA provides up to 33 and 7 built-in models, respectively, while all the models provide quick training and instant inference. Besides, PYABSA contains to 180K+ ABSA examples from 21 augmented ABSA datasets for applications and studies.PYABSA is available at https://github. com/yangheng95/PyABSA.1 We provides an ATESC inference service demo on https://huggingface.co/spaces/yangheng/Multilingual-Aspect-Based-Sentiment-Analysis 2 We refer to ASC as aspect polarity classification (APC) in PYABSA's implementation.3 Our Fast-LSA model and LCF-ATEPC model achieve state-of-the-art performance on ASC subtask.

show abstract

“…Sentiment-enhanced word embedding To solve the above problems, we expect to find a method to build the mapping relationship between words and their sentiment orientations in sentiment lexicons and fuse sentiment information into these words. Fortunately, we are inspired by translations in the embedding space (TransE) [19] and contrastive learning [20,21]. TransE is a knowledge graph embedding method.…”

Section: Sentiment Orientationmentioning

confidence: 99%

“…Meanwhile, contrastive learning narrows the distance between positive examples and increases the distance between negative examples in vector space. The bootstrap your own latent (BYOL) [20] minimizes the distance between two similar images; the supervised contrastive pre-training (SCAPT) [21] used supervised contrastive learning to cluster explicit and implicit positive sentiments, cluster explicit and implicit negative sentiments, and separated these two clusters. By borrowing these ideas, we propose a novel sentiment-enhanced word embedding (S-EWE) method to improve the performances of sentiment classification models.…”

Section: Sentiment Orientationmentioning

confidence: 99%

A New Sentiment-Enhanced Word Embedding Method for Sentiment Analysis

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

Since some sentiment words have similar syntactic and semantic features in the corpus, existing pre-trained word embeddings always perform poorly in sentiment analysis tasks. This paper proposes a new sentiment-enhanced word embedding (S-EWE) method to improve the effectiveness of sentence-level sentiment classification. This sentiment enhancement method takes full advantage of the mapping relationship between word embeddings and their corresponding sentiment orientations. This method first converts words to word embeddings and assigns sentiment mapping vectors to all word embeddings. Then, word embeddings and their corresponding sentiment mapping vectors are fused to S-EWEs. After reducing the dimensions of S-EWEs through a fully connected layer, the predicted sentiment orientations are obtained. The S-EWE method adopts the cross-entropy function to calculate the loss between predicted and true sentiment orientations, and backpropagates the loss to train the sentiment mapping vectors. Experiments show that the accuracy and macro-F1 values of six sentiment classification models using Word2Vec and GloVe with the S-EWEs are on average 1.07% and 1.58% higher than those without the S-EWEs on the SemEval-2013 dataset, and on average 1.23% and 1.26% higher than those without the S-EWEs on the SST-2 dataset. In all baseline models with S-EWEs, the convergence time of the attention-based bidirectional CNN-RNN deep model (ABCDM) with S-EWEs was significantly decreased by 51.21% of ABCDM on the SemEval-2013 dataset. The convergence time of CNN-LSTM with S-EWEs was vastly reduced by 41.34% of CNN-LSTM on the SST-2 dataset. In addition, the S-EWE method is not valid for contextualized word embedding models. The main reasons are that the S-EWE method only enhances the embedding layer of the models and has no effect on the models themselves.

show abstract

Learning Implicit Sentiment in Aspect-based Sentiment Analysis with Supervised Contrastive Pre-Training

Cited by 63 publications

References 31 publications

A Transfer Learning Method for Detecting Alzheimer's Disease Based on Speech and Natural Language Processing

A Transfer Learning Method for Detecting Alzheimer's Disease Based on Speech and Natural Language Processing

PyABSA: A Modularized Framework for Reproducible Aspect-based Sentiment Analysis

A New Sentiment-Enhanced Word Embedding Method for Sentiment Analysis

Contact Info

Product

Resources

About