Investigation of Various Hybrid Acoustic Modeling Units via a Multitask Learning and Deep Neural Network Technique for LVCSR of the Low-Resource Language, Amharic

Abstract

Multitask learning (MTL) is helpful for improving the performance of related tasks when the training dataset is limited and sparse, especially for low-resource languages. Amharic is a lowresource language and suffers from the problems of training data scarcity, sparsity, and unevenness. Consequently, fundamental acoustic units-based speech recognizers perform worse compared with the speech recognizers of technologically favored languages. This paper presents the results of our contributions to the use of various hybrid acoustic modeling units for the Amharic language. The fundamental acoustic units, namely, syllable, phone, and rounded phone units-based deep neural network (DNN) models have been developed. Various hybrid acoustic units have been investigated by jointly training the fundamental acoustic units via the MTL technique. Those hybrid units and the fundamental units are discussed and compared. The experimental results demonstrate that all the fundamental units-based DNN models outperform the Gaussian mixture models (GMM) with relative performance improvements of 14.14%-23.31%. All the hybrid units outperform the fundamental acoustic units with relative performance improvements of 1.33%-4.27%. The syllable and phone units exhibit higher performance under sufficient and limited training datasets, respectively. All the hybrid units are useful with both sufficient and limited training datasets and outperformed the fundamental units. Overall, our results show that DNN is an effective acoustic modeling technique for the Amharic language. The context-dependent (CD) syllable is the more suitable unit if a sufficient training corpus is available and the accuracy of the recognizer is prioritized. The CD phone is a superior unit if the available training dataset is limited and realizes the highest accuracy and fast recognition speed. The hybrid acoustic units perform the best under both sufficient and limited training datasets and achieve the highest accuracy. INDEX TERMS Acoustic modeling units, Amharic, hybrid acoustic modeling units, low-resource language, multitask learning.

Acoustic Modeling Based on Deep Learning for Low-Resource Speech Recognition: An Overview

“…Chen et al [91] took grapheme modeling as an additional learning task and used multi-task learning DNN to learn the phone model of the target language. Fantaye et al [92] studied to conduct joint training through multi-task learning for basic acoustic units of a lowresource language Amharic, including syllable, phone and rounded phone.…”

Advanced Convolutional Neural Network-Based Hybrid Acoustic Models for Low-Resource Speech Recognition

“…The DNN model is an old feed-forward network that has been used by several researchers in low-resource speech recognitions. For example, Fantaye et al [7], Sercu et al [34], Sriranjani et al [8], and Chan W. and Lane I. [16] used DNN model for Amharic, Cebuano, Hindi, and Bengali languages, respectively.…”

“…Moreover, we used two Ethiopic languages, namely Amharic and Chaha, as additional target languages for evaluating our proposed models. Amharic (https://github.com/besacier/ALFFA_PUBLIC/tree/master/ASR/AMHARIC) is a low-resource language, which has only 26-h read speech training corpus [7]. Chaha (https://m.scirp.org/papers/97733) is a very low-resource language that has only 2.67-h read speech training corpus [42].…”

“…The description of the training and development datasets of the target languages are given in Table 5. The word-based lexical dictionaries of the Babel languages were prepared by expanding the released lexicons based on the language-specific peculiarities documents from the transcribed training texts, while the word-based lexicons of Amharic and Chaha were prepared using grapheme-based approach by taking the most frequent words from the text corpus of 216,076 sentences [7] and 14,595 sentences [42], respectively. The word-based backoff trigram statistical language models were also developed using the SRILM toolkit [62].…”

“…In hybrid acoustic modeling, various deep neural network models, such as feed-forward networks (fully connected deep neural network (DNN) [1][2][3][4][5][6][7][8] and convolutional neural network (CNN) [9][10][11][12][13][14][15][16]) and deep recurrent neural networks (conventional recurrent neural network (RNN) [17], long short-term memory (LSTM) [18][19][20][21][22], and gated recurrent unit (GRU) [23][24][25][26]) are examined for both low-and high-resource-languages speech recognition tasks. These neural network models have strong and weak sides.…”