Combining BERT with TCN-BiGRU for enhancing Arabic aspect category detection

Bensoltane, Rajae; Zaki, Taher

doi:10.3233/jifs-221214

Cited by 9 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second task allows the model to predict whether a sentence is the next sentence in a given sequence of sentences. The BERT model has improved the results of many NLP tasks including named entity recognition [33], [34] text classification [35], [36] and sentiment analysis [37], [38]. Figure 2 illustrates the architecture of the BERT model.…”

Section: Bidirectional Encoder Representations From Transformers Modelmentioning

confidence: 99%

Enhancing Arabic offensive language detection with BERT-BiGRU model

Bensoltane,

Zaki

2024

Bulletin EEI

View full text Add to dashboard Cite

With the advent of Web 2.0, various platforms and tools have been developed to allow internet users to express their opinions and thoughts on diverse topics and occurrences. Nevertheless, certain users misuse these platforms by sharing hateful and offensive speeches, which has a negative impact on the mental health of internet society. Thus, the detection of offensive language has become an active area of research in the field of natural language processing. Rapidly detecting offensive language on the internet and preventing it from spreading is of great practical significance in reducing cyberbullying and self-harm behaviors. Despite the crucial importance of this task, limited work has been done in this field for nonEnglish languages such as Arabic. Therefore, in this paper, we aim to improve the results of Arabic offensive language detection without the need for laborious preprocessing or feature engineering work. To achieve this, we combine the bidirectional encoder representations from transformers (BERT) model model with a bidirectional gated recurrent unit (BiGRU) layer to further enhance the extracted context and semantic features. The experiments were conducted on the Arabic dataset provided by the SemEval 2020 Task 12. The evaluation results show the effectiveness of our model compared to the baseline and related work models by achieving a macro F1- score of 93.16%.

show abstract

Section: Bidirectional Encoder Representations From Transformers Modelmentioning

confidence: 99%

Enhancing Arabic offensive language detection with BERT-BiGRU model

Bensoltane,

Zaki

2024

Bulletin EEI

View full text Add to dashboard Cite

show abstract

“…Logging data at depths ranging from 1621 to 1710 m are used as the training data to build the models, including the back propagation neural network (BPNN) [20], GRU [21], BiGRU-AM [22], and ISCSO-TCN-BiGRU-AM. The training models take six parameters as inputs: DEPTH, CAL, NPHI, GR, DT, and RT.…”

Section: Model Parameter Settingmentioning

confidence: 99%

Well Logging Reconstruction Based on a Temporal Convolutional Network and Bidirectional Gated Recurrent Unit Network with Attention Mechanism Optimized by Improved Sand Cat Swarm Optimization

Wang,

Teng,

Qiao

et al. 2024

Energies

View full text Add to dashboard Cite

Geophysical logging plays a very important role in reservoir evaluation. In the actual production process, some logging data are often missing due to well wall collapse and instrument failure. Therefore, this paper proposes a logging reconstruction method based on improved sand cat swarm optimization (ISCSO) and a temporal convolutional network (TCN) and bidirectional gated recurrent unit network with attention mechanism (BiGRU-AM). The ISCSO-TCN-BiGRU-AM can process both past and future states efficiently, thereby extracting valuable deterioration information from logging data. Firstly, the sand cat swarm optimization (SCSO) improved by the variable spiral strategy and sparrow warning mechanism is introduced. Secondly, the ISCSO’s performance is evaluated using the CEC–2022 functions and the Wilcoxon test, and the findings demonstrate that the ISCSO outperforms the rival algorithms. Finally, the logging reconstruction method based on the ISCSO-TCN-BiGRU-AM is obtained. The results are compared with the competing models, including the back propagation neural network (BPNN), GRU, and BiGRU-AM. The results show that the ISCSO-TCN-BiGRU-AM has the best performance, which verifies its high accuracy and feasibility for the missing logging reconstruction.

show abstract

“…Other transformer models like generative pre-trained transformer (GPT) and efficiently learning an encoder that classifies token replacements accurately (ELECTRA) are under-explored in this area. Moreover, the impact of incorporating these models with more complex neural network layers has been investigated in many tasks like aspect term extraction [24], named entity recognition [25], and aspect category detection [26]. Yet further efforts are required to explore the effectiveness of this combined approach for the task of hate speech detection.…”

Section: Introductionmentioning

confidence: 99%

Fine-grained hate speech detection in Arabic using transformer-based models

Bensoltane,

Zaki

2024

IJECE

View full text Add to dashboard Cite

With the proliferation of social media platforms, characterized by features such as anonymity, user-friendly access, and the facilitation of online community building and discourse, the matter of detecting and monitoring hate speech has emerged as an increasingly formidable challenge for society, individuals, and researchers. Despite the crucial importance of hate speech detection task, the majority of work in this field has been conducted in English, with insufficient focus on other languages, particularly Arabic. Furthermore, most existing studies on Arabic hate speech detection have addressed this task as a binary classification problem, which is unreliable. Therefore, the aim of this study is to provide an enhanced model for detecting fine-grained hate speech in Arabic. To this end, three transformer-based models were evaluated to generate contextualized word embeddings from input sequence. Additionally, these models were combined with a bidirectional gated recurrent unit (BiGRU) layer to further improve the extracted semantic and context features. The experiments were conducted on an Arabic reference dataset provided by the open-source Arabic corpora and processing tools (OSACT-5) shared task. A comparative analysis indicates the efficiency of the proposed model over the baseline and related work models by achieving a macro F1-score of 61.68%.

show abstract

Combining BERT with TCN-BiGRU for enhancing Arabic aspect category detection

Cited by 9 publications

References 11 publications

Enhancing Arabic offensive language detection with BERT-BiGRU model

Enhancing Arabic offensive language detection with BERT-BiGRU model

Well Logging Reconstruction Based on a Temporal Convolutional Network and Bidirectional Gated Recurrent Unit Network with Attention Mechanism Optimized by Improved Sand Cat Swarm Optimization

Fine-grained hate speech detection in Arabic using transformer-based models

Contact Info

Product

Resources

About