YASO: A Targeted Sentiment Analysis Evaluation Dataset for Open-Domain Reviews

Orbach, Matan; Toledo-Ronen, Orith; Spector, Artem; Aharonov, Ranit; Katz, Yoav; Slonim, Noam

doi:10.18653/v1/2021.emnlp-main.721

Cited by 13 publications

(13 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, ATE methods have focused on neural networks such as LSTM (Liu et al, 2015), CNN (Xu et al, 2018) or Transformer ). An ATE system can be used in downstream applications such as sentiment analysis Pouran Ben Veyseh et al, 2020b;Orbach et al, 2021) or opinion term extraction (Pouran Ben Veyseh et al, 2020a). One of the challenges for this task is the scarcity of training data which hinders the training of large neural networks.…”

Section: Related Workmentioning

confidence: 99%

Generating Complement Data for Aspect Term Extraction with GPT-2

Veyseh¹,

Dernoncourt²,

Nguyen³

2022

Proceedings of the Third Workshop on Deep Learning for Low-Resource Natural Language Processing

View full text Add to dashboard Cite

Aspect Term Extraction (ATE) is the task of identifying the word(s) in a review text toward which the author express an opinion. A major challenges for ATE involve data scarcity that hinder the training of deep sequence taggers to identify rare targets. To overcome these issues, we propose a novel method to better exploit the available labeled data for ATE by computing effective complement sentences to augment the input data and facilitate the aspect term prediction. In particular, we introduce a multistep training procedure that first obtains optimal complement representations and sentences for training data with respect to a deep ATE model. Afterward, we fine-tune the generative language model GPT-2 to allow complement sentence generation at test data. The REIN-FORCE algorithm is employed to incorporate different expected properties into the reward function to perform the fine-tuning. We perform extensive experiments on the benchmark datasets to demonstrate the benefits of the proposed method that achieve the state-of-the-art performance on different datasets.

show abstract

Section: Related Workmentioning

confidence: 99%

Generating Complement Data for Aspect Term Extraction with GPT-2

Veyseh¹,

Dernoncourt²,

Nguyen³

2022

Proceedings of the Third Workshop on Deep Learning for Low-Resource Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…This extra annotation layer of the YASO evaluation data is available online (see §1). As suggested in Orbach et al (2021), YASO is used solely for evaluation. MAMS Jiang et al (2019) collected the MAMS dataset over restaurant reviews.…”

Section: Generating Weak Labelsmentioning

confidence: 99%

“…6 The reviews were then annotated for TSA by asking annotators to mark all sentiment-bearing targets in each sentence. This step is similar to the candidates annotation phase described in Orbach et al (2021). However, unlike in our previous work, the detected candidates we collected were not passed through another verification step, to reduce costs.…”

Section: Diversifying the Training Setmentioning

confidence: 99%

“…However, obtaining such a dataset is a challenge. The existing TSA datasets are limited in their diversity, and the collection of a new large scale diverse TSA dataset is complex (Orbach et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

“…This dataset is limited to two domains: restaurants or laptops. Each model went through several selftraining iterations and evaluated on three TSA publicly available datasets: SE, the MAMS dataset of restaurant reviews (Jiang et al, 2019), and the YASO dataset of open-domain reviews (Orbach et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-Domain Targeted Sentiment Analysis

Toledo-Ronen¹,

Orbach²,

Katz³

et al. 2022

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

Targeted Sentiment Analysis (TSA) is a central task for generating insights from consumer reviews. Such content is extremely diverse, with sites like Amazon or Yelp containing reviews on products and businesses from many different domains. A real-world TSA system should gracefully handle that diversity. This can be achieved by a multi-domain model -one that is robust to the domain of the analyzed texts, and performs well on various domains. To address this scenario, we present a multi-domain TSA system based on augmenting a given training set with diverse weak labels from assorted domains. These are obtained through self-training on the YELP reviews corpus. Extensive experiments with our approach on three evaluation datasets across different domains demonstrate the effectiveness of our solution. We further analyze how restrictions imposed on the available labeled data affect the performance, and compare the proposed method to the costly alternative of manually gathering diverse TSA labeled data. Our results and analysis show that our approach is a promising step towards a practical domain-robust TSA system.

show abstract