2020
DOI: 10.48550/arxiv.2012.15516
|View full text |Cite
Preprint
|
Sign up to set email alerts
|

AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1
1
1

Citation Types

0
16
0

Year Published

2021
2021
2024
2024

Publication Types

Select...
4
3
1

Relationship

0
8

Authors

Journals

citations
Cited by 11 publications
(16 citation statements)
references
References 0 publications
0
16
0
Order By: Relevance
“…This can be attributed to the large number of datasets published at the bi-annual LREC conference. We also anticipate a significant increase, particularly in 2020, with the emergence of pretrained language models language models namely AraBERT (Antoun et al, 2020a), Multi-dialect BERT (Talafha et al, 2020) and Araelectra (Antoun et al, 2020b). We can also observe that most of the datasets are published in conferences and workshops.…”
Section: Publications Developmentmentioning
confidence: 85%
“…This can be attributed to the large number of datasets published at the bi-annual LREC conference. We also anticipate a significant increase, particularly in 2020, with the emergence of pretrained language models language models namely AraBERT (Antoun et al, 2020a), Multi-dialect BERT (Talafha et al, 2020) and Araelectra (Antoun et al, 2020b). We can also observe that most of the datasets are published in conferences and workshops.…”
Section: Publications Developmentmentioning
confidence: 85%
“…Further, the discriminator does not predict the original tokens, instead, it predicts whether each token in the corrupted input was replaced with a generator sample or not [57]. AraElectra [58] is the Arabic variant of ELECTRA model. It was pretrained on a large Arabic text using Replaced token detection and was evaluated by different Arabic NLP tasks such as sentiment analysis and named entity recognition(NER).…”
Section: Ai Techniques For Arabic Sarcasm Detectionmentioning
confidence: 99%
“…Further, the dataset was split into 90-10% for training and test sets respectively. Regarding the experiment, they applied eight variants of two transformer-based models, which are AraElECTRA [58] and AraBERT [20]. Finally, the models were stacked to get the best performance, which was measured using precision, recall and macro F1-score score giving the following results: 72.64, 71.47 and 72.00, respectively.…”
Section: Literature Reviewmentioning
confidence: 99%
“…Usually, a discriminator is taken and fine-tuned for downstream tasks. The Arabic-specific ELECTRA provided by Antoun et al ( 2020b ), was trained on the same textual corpus used for AraBERT. However, we noticed that AraELECTRA is one of the top performing models while being much more efficient in its computational cost and achieves similar results to AraBERT that has doubled the number of parameters.…”
Section: Emotion Analysis Of Arabic Tweetsmentioning
confidence: 99%