Improvements in Serbian Speech Recognition Using Sequence-Trained Deep Neural Networks

Abstract

Improvements in Serbian Speech Recognition using Sequence-Trained Deep Neural Networks.Abstract. This paper presents the recent improvements in Serbian speech recognition that were obtained by using contemporary deep neural networks based on sequence-discriminative training to train robust acoustic models. More specifically, several variants of the new large vocabulary continuous speech recognition (LVCSR) system are described, all based on the lattice-free version of the maximum mutual information (LF-MMI) training criterion. The parameters of the system were varied to achieve best possible word error rate (WER) and character error rate (CER), using the largest speech database for Serbian in existence and the best n-gram based language model made for general purposes. In addition to tuning the neural network itself (its layers, complexity, layer splicing etc.) other language-specific optimizations were explored, such as the usage of accent-specific vowel phoneme models, and its combination with pitch features to produce the best possible results. Finally, speech database tuning was tested as well. Artificial database expansion was made by modifying speech speed in utterances, as well as volume scaling in an attempt to improve speech variability.The results showed that 8-layer deep neural network with 625-neuron layers works best in the given environment, without the need for speech database augmentation or volume adjustments, and that pitch features in combination with the introduction of accented vowel models provide the best performance out of all experiments.

Speech Technology Progress Based on New Machine Learning Paradigm

“…in [74] developed lattice-free maximum mutual information using phone n -gram language model starting from randomly initialized neural networks. This method was also successfully applied to Serbian [75]; i.e., the relative reduction of WER was about 25% with respect to the best previous system.…”

Using Morphological Data in Language Modeling for Serbian Large Vocabulary Speech Recognition

“…The used acoustic models were subsampled time-delay neural networks (TDNNs), which are trained using cross-entropy training within the so-called “chain” training method [17]. For this purpose, the Kaldi speech recognition toolkit [18] was used.…”

“…Alignments for the deep neural network (DNN) training were provided by a previously trained speaker-adaptive HMM-GMM (hidden Markov model—Gaussian mixture model) system [19] with 3500 states and 35000 Gaussians. Acoustic features used for DNN training were 40 high-resolution MFCC features (Mel-frequency cepstral coefficients), alongside their first- and second-order derivatives, as well as 3 pitch-based features—weighted log-pitch, delta-log-pitch, and warped normalized cross-correlation function (NCCF) value (which is originally between −1 and 1, and higher for voiced frames), and their derivatives, producing a 129-dimensional feature vector, which is a configuration already used in other experiments [5, 15, 17]. The context dependency tree used for the “chain” training with its special model topology that allows a subsampling factor of 3 had 2000 leaves (output states).…”

Transfer learning for domain and environment adaptation in Serbian ASR

“…Different recurrent neural network based language models were trained and tested, as well as variants that use embedding vectors as word representations and incorporate additional lexical and morphological features [1] - [2]. On the other hand, the latest acoustic models involved purely sequence-trained deep neural networks with subsampling, specifically designed to better model longer temporal contexts [3] - [4]. These models included accent-specific vowel models, Melfrequency cepstral coefficients (MFCCs), pitch features and i-vectors [5] for the purpose of adaptation to different speakers and channels.…”