Assessing the impact of minor modifications on the interior structure of GRU: GRU1 and GRU2

Abstract

In this study, two GRU variants named GRU1 and GRU2 are proposed by employing simple changes to the internal structure of the standard GRU, which is one of the popular RNN variants. Comparative experiments are conducted on four problems: language modeling, question answering, addition task, and sentiment analysis. Moreover, in the addition task, curriculum learning and anti-curriculum learning strategies, which extend the training data having examples from easy to hard or from hard to easy, are comparatively evaluated. Accordingly, the GRU1 and GRU2 variants outperformed the standard GRU. In addition, the curriculum learning approach, in which the training data is expanded from easy to difficult, improves the performance considerably.

Special Issue on International Conference on Innovations in Intelligent Systems and Applications (INISTA 2021)

“…The authors propose two GRU variants with minor changes in the interior structure of the GRU to achieve better performances than the standard GRU in question answering, sentiment analysis, language modeling, and addition task problems. 3 Nevena Rankovic, Dragica Rankovic, Mirjana Ivanovic, and Ljubomir Lazic, "Influence of input values on the prediction model error using artificial neural network based on Taguchi's Orthogonal Array." One of the most difficult issues and duties in software organizations is estimating effort and costs for completing projects.…”

“…One of the most popular variants of RNNs that address these problems is gated recurrent networks (GRU) which is commonly used for many tasks such as language modeling, question answering, speech recognition, sentiment classification, machine translation, image captioning and so forth. The authors propose two GRU variants with minor changes in the interior structure of the GRU to achieve better performances than the standard GRU in question answering, sentiment analysis, language modeling, and addition task problems 3 …”