Exploring BERT for Aspect Extraction in Portuguese Language

Lopes, Émerson; Corrêa, Ulisses Brisolara; Freitas, Larissa Astrogildo de

doi:10.32473/flairs.v34i1.128357

Cited by 10 publications

(8 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Here, a fine-tuned BERT model was employed to extract aspects from the multi-domain datasets. Moreover, Lopes et al [40] utilized BERT for the ATE in Portuguese. Winatmoko et al [41] extracted aspect terms from Bahasa Indonesian hotel reviews using the multilingual BERT, extra auxiliary label, and CRF as the output layer, and the results demonstrated improved performance.…”

Section: Aspect Term Extractionmentioning

confidence: 99%

Multi-Task Learning Model with Data Augmentation for Arabic Aspect-Based Sentiment Analysis

Fadel¹,

Abulnaja²,

Saleh³

2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Aspect-based sentiment analysis (ABSA) is a fine-grained process. Its fundamental subtasks are aspect term extraction (ATE) and aspect polarity classification (APC), and these subtasks are dependent and closely related. However, most existing works on Arabic ABSA content separately address them, assume that aspect terms are preidentified, or use a pipeline model. Pipeline solutions design different models for each task, and the output from the ATE model is used as the input to the APC model, which may result in error propagation among different steps because APC is affected by ATE error. These methods are impractical for real-world scenarios where the ATE task is the base task for APC, and its result impacts the accuracy of APC. Thus, in this study, we focused on a multi-task learning model for Arabic ATE and APC in which the model is jointly trained on two subtasks simultaneously in a single model. This paper integrates the multi-task model, namely Local Cotext Foucse-Aspect Term Extraction and Polarity classification (LCF-ATEPC) and Arabic Bidirectional Encoder Representation from Transformers (AraBERT) as a shred layer for Arabic contextual text representation. The LCF-ATEPC model is based on a multi-head selfattention and local context focus mechanism (LCF) to capture the interactive information between an aspect and its context. Moreover, data augmentation techniques are proposed based on state-of-the-art augmentation techniques (word embedding substitution with constraints and contextual embedding (AraBERT)) to increase the diversity of the training dataset. This paper examined the effect of data augmentation on the multi-task model for Arabic ABSA. Extensive experiments were conducted on the original and combined datasets (merging the original and augmented datasets). Experimental results demonstrate that the proposed Multi-task model outperformed existing APC techniques. Superior results were obtained by AraBERT and LCF-ATEPC with fusion layer (AR-LCF-ATEPC-Fusion) and the proposed data augmentation word embedding-based method (FastText) on the combined dataset.

show abstract

Section: Aspect Term Extractionmentioning

confidence: 99%

Multi-Task Learning Model with Data Augmentation for Arabic Aspect-Based Sentiment Analysis

Fadel¹,

Abulnaja²,

Saleh³

2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…In the literature, we find some works about SA (Document and Aspect-level) in the Portuguese language; they are Barros (2021), Lopes, Corrêa, and Freitas (2021), Lopes et al (2022), da Silva et al (2022), Gomes et al (2022). Also, we find works about HS, as: Pelle and Moreira (2017), Fortuna et al (2019), Leite et al (2020), Silva and Freitas (2022) and, we find works about ID, as: Corrêa et al (2021), Subies (2021), Jiang et al (2021).…”

Section: Related Workmentioning

confidence: 99%

BERTimbau in Action: An Investigation of its Abilities in Sentiment Analysis, Aspect Extraction, Hate Speech Detection, and Irony Detection.

Junqueira

Silva

Costa

et al. 2023

FLAIRS

View full text Add to dashboard Cite

Social Media has revolutionized how individuals, groups, and communities interact. This immense quantity of unstructured data holds valuable information expressed in informal language. However, automatically extracting this information using Natural Language Processing requires adaptations of traditional methods or the development of new strategies capable of extracting information tackling web-prone language. BERT, a Deep Learning methodology proposed by Google in 2018, brought transfer learning to Natural Language Processing. In this work, we used a BERT model for the Portuguese language called BERTimbau to create models for Sentiment Analysis, Aspect Extraction, Hate Speech Detection, and Irony Detection. We experimented with the two BERTimbau models, base and large. Finally, we compared the results obtained in each task. Experiments with BERTimbau based models obtained improved results, F-Measure of 0.88 and 0.89 in Sentiment Analysis and Hate Speech Detection tasks, respectively, compared to classical Machine Learning approaches.

show abstract

“…The results showed improvement in the F1 score. Lopes et al [49] also used BERT for an AE task on a Portuguese dataset.…”

Section: A English and Other Languagesmentioning

confidence: 99%

Arabic Aspect Extraction Based on Stacked Contextualized Embedding With Deep Learning

2022

View full text Add to dashboard Cite

The exponential growth of the internet and a multi-fold increase in social media users in the last decade have resulted in a massive growth of unstructured data. Aspect-Based Sentiment Analysis (ABSA) is challenging because it performs a fine-grain analysis; it is a text analysis technique where the opinions group is based on the aspect. The Aspect Extraction (AE) task is one of the core subtasks of ABSA; it helps to identify aspect terms in the text, comments, or reviews. The challenge of the Arabic AE task increases due to the complexity of the Arabic language. This work aims to develop the Arabic AE task by proposing transfer learning using State-of-art pre-trained contextual language models. We concatenate the Bidirectional Encoder Representation from Transformers (BERT) language model and contextualize string embedding (Flair embedding) as a stacked embeddings layer for better word representation for Arabic language. Then, we extend it with different deep learning network architectures. For Arabic AE, the model is developed by concatenating the Arabic contextual language model, AraBERT, and Flair embedding as a contextual stacked embeddings layer with an extended layer, BiLSTM-CRF or BiGRU-CRF, for sequence labeling. Our proposed models are called BF-BiLSTM-CRF and BF-BiGRU-CRF. The proposed model is evaluated using the Arabic Hotel's reviews dataset. For performance evaluation, we used the F1 score. The experimental results show that the proposed BF-BiLSTM-CRF configuration outperformed the baseline and other models by achieving an F1score of 79.7%.

show abstract

Exploring BERT for Aspect Extraction in Portuguese Language

Cited by 10 publications

References 26 publications

Multi-Task Learning Model with Data Augmentation for Arabic Aspect-Based Sentiment Analysis

Multi-Task Learning Model with Data Augmentation for Arabic Aspect-Based Sentiment Analysis

BERTimbau in Action: An Investigation of its Abilities in Sentiment Analysis, Aspect Extraction, Hate Speech Detection, and Irony Detection.

Arabic Aspect Extraction Based on Stacked Contextualized Embedding With Deep Learning

Contact Info

Product

Resources

About