A simple Galois Power-of-Two real time embedding scheme for performing Arabic morphology deep learning tasks

ElAffendi, Mohammed; Abuhaimed, Ibrahim; Alrajhi, Khawla

doi:10.1016/j.eij.2020.03.002

Cited by 6 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PIA attention vectors are computed using the reverse integer transformation algorithm (RIT) [ 43 , 44 ]. The main RIT algorithm is described below.…”

Section: A Novel Polynomial Inherent Attention (Pia) Model and Its Im...mentioning

confidence: 99%

Beyond the Transformer: A Novel Polynomial Inherent Attention (PIA) Model and Its Great Impact on Neural Machine Translation

ElAffendi

Alrajhi

2022

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

This paper describes a novel polynomial inherent attention (PIA) model that outperforms all state-of-the-art transformer models on neural machine translation (NMT) by a wide margin. PIA is based on the simple idea that natural language sentences can be transformed into a special type of binary attention context vectors that accurately capture the semantic context and the relative dependencies between words in a sentence. The transformation is performed using a simple power-of-two polynomial transformation that maintains strict consistent positioning of words in the resulting vectors. It is shown how this transformation reduces the neural machine translation process to a simple neural polynomial regression model that provides excellent solutions to the alignment and positioning problems haunting transformer models. The test BELU scores obtained on the WMT-2014 data set are 75.07 BELU for the EN-FR data set and 66.35 BELU for the EN-DE data set—well above accuracies achieved by state-of-the-art transformer models for the same data sets. The improvements are, respectively, 65.7% and 87.42%.

show abstract

“…PIA attention vectors are computed using the reverse integer transformation algorithm (RIT) [ 43 , 44 ]. The main RIT algorithm is described below.…”

Section: A Novel Polynomial Inherent Attention (Pia) Model and Its Im...mentioning

confidence: 99%

Beyond the Transformer: A Novel Polynomial Inherent Attention (PIA) Model and Its Great Impact on Neural Machine Translation

ElAffendi

Alrajhi

2022

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

show abstract

“…where y is a power-of-two radix (y is 2 n ) and ci is a binary representation of the respective character. PSPA has been implemented using a simple Galois powerof-two positional binary embedding scheme (PBES) [50], [51], where words are transformed into base-64 binary integers (radix= 64=2 6 ), and sentences are transformed into base-2 60 binary integers.…”

Section: The Polynomial Space Positioning Attention (Pspa) Modelmentioning

confidence: 99%

A Novel Deep Learning-Based Multilevel Parallel Attention Neural (MPAN) Model for Multidomain Arabic Sentiment Analysis

2021

Self Cite

View full text Add to dashboard Cite

“…Pointwise mutual information is also recorded [20] for the words joint by 'AND' word which show that words occurring multiple times together have higher association. ELAffendi et al [26] have recently used a neural network based approach to predict the morphological patterns and POS tags for Arabic. Galois Power-of-Two (GPOW2) calculates the real time embeddings for character, word and sentence in parallel.…”

Section: A Grammatical Nlp Analysismentioning

confidence: 99%

Arabic Natural Language Processing for Qur’anic Research: A Systematic Review

Bashir¹,

Azmi²,

Nawaz³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

A simple Galois Power-of-Two real time embedding scheme for performing Arabic morphology deep learning tasks

Cited by 6 publications

References 6 publications

Beyond the Transformer: A Novel Polynomial Inherent Attention (PIA) Model and Its Great Impact on Neural Machine Translation

Beyond the Transformer: A Novel Polynomial Inherent Attention (PIA) Model and Its Great Impact on Neural Machine Translation

A Novel Deep Learning-Based Multilevel Parallel Attention Neural (MPAN) Model for Multidomain Arabic Sentiment Analysis

Arabic Natural Language Processing for Qur’anic Research: A Systematic Review

Contact Info

Product

Resources

About