Generative adversarial networks for speech processing: A review

Wali, Aamir; Alamgir, Zareen; Karim, Saira; Fawaz, Ather; Ali, Mubariz Barkat; Adan, Muhammad; Mujtaba, Malik

doi:10.1016/j.csl.2021.101308

Cited by 47 publications

(23 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the existing models of lexical learning focus primarily on either ASR/speech-to-text (perception) or text-tospeech/speech synthesis (production; see [3] for an overview). Variational Autoencoders (VAEs) involve both an encoder and decoder, which allows unsupervised acoustic word embedding as well as generation of speech, but these proposals only use VAEs for either unsupervised ASR [4,5,6,7] or for speech synthesis/transformation (e.g.…”

Section: Prior Workmentioning

confidence: 99%

Modeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no access to speech data

Begus

2022

View full text Add to dashboard Cite

Human speakers encode information into raw speech which is then decoded by the listeners. This complex relationship between encoding (production) and decoding (perception) is often modeled separately. Here, we test how encoding and decoding of lexical semantic information can emerge automatically from raw speech in unsupervised generative deep convolutional networks that combine the production and perception principles of speech. We introduce, to our knowledge, the most challenging objective in unsupervised lexical learning: a network that must learn unique representations for lexical items with no direct access to training data. We train several models (ciwGAN and fiwGAN [1]) and test how the networks classify acoustic lexical items in unobserved test data. Strong evidence in favor of lexical learning and a causal relationship between latent codes and meaningful sublexical units emerge. The architecture that combines the production and perception principles is thus able to learn to decode unique information from raw acoustic data without accessing real training data directly. We propose a technique to explore lexical (holistic) and sublexical (featural) learned representations in the classifier network. The results bear implications for unsupervised speech technology, as well as for unsupervised semantic modeling as language models increasingly bypass text and operate from raw acoustics.

show abstract

Section: Prior Workmentioning

confidence: 99%

Modeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no access to speech data

Begus

2022

View full text Add to dashboard Cite

show abstract

“…The majority of popular ASR systems use Gaussian Mixture Models (GMMs), Hidden Markov Models (HMMs), and Deep Neural Networks (DNNs) [ 12 , 13 , 14 , 15 ]. DNNs play an essential part in the building of ASR systems [ 16 , 17 ], mostly because of the evolution of unique neural network models, as well as training and classification techniques [ 18 , 19 ]. They have also been applied to problems such as feature extraction [ 20 , 21 ], audio signal classification [ 22 , 23 ], text recognition and TTS [ 24 , 25 ], disordered speech processing [ 26 ], and speech recognition based on small and large vocabulary [ 27 ].…”

Section: Introductionmentioning

confidence: 99%

Automatic Speech Recognition Method Based on Deep Learning Approaches for Uzbek Language

Mukhamadiyev

Khujayorov

Djuraev

et al. 2022

Sensors

View full text Add to dashboard Cite

Communication has been an important aspect of human life, civilization, and globalization for thousands of years. Biometric analysis, education, security, healthcare, and smart cities are only a few examples of speech recognition applications. Most studies have mainly concentrated on English, Spanish, Japanese, or Chinese, disregarding other low-resource languages, such as Uzbek, leaving their analysis open. In this paper, we propose an End-To-End Deep Neural Network-Hidden Markov Model speech recognition model and a hybrid Connectionist Temporal Classification (CTC)-attention network for the Uzbek language and its dialects. The proposed approach reduces training time and improves speech recognition accuracy by effectively using CTC objective function in attention model training. We evaluated the linguistic and lay-native speaker performances on the Uzbek language dataset, which was collected as a part of this study. Experimental results show that the proposed model achieved a word error rate of 14.3% using 207 h of recordings as an Uzbek language training dataset.

show abstract

“…Generally, pattern recognition systems consist of two main components: feature analysis and pattern classification. Most state-of-the-art speech recognition systems are based on hidden Markov models (HMMs) or artificial neural networks (ANNs), or HMM and ANN hybrids [ 12 , 13 , 14 , 15 ]. Neural networks play an important role both in speech [ 15 , 16 , 17 ] and speaker recognition [ 18 , 19 , 20 , 21 ], mainly due to the development of new neural network topologies as well as training and classification algorithms [ 14 , 22 , 23 ].…”

Section: Introductionmentioning

confidence: 99%

“…Most state-of-the-art speech recognition systems are based on hidden Markov models (HMMs) or artificial neural networks (ANNs), or HMM and ANN hybrids [ 12 , 13 , 14 , 15 ]. Neural networks play an important role both in speech [ 15 , 16 , 17 ] and speaker recognition [ 18 , 19 , 20 , 21 ], mainly due to the development of new neural network topologies as well as training and classification algorithms [ 14 , 22 , 23 ]. They have also been used for tasks such as classification [ 12 , 24 , 25 ] or feature extraction [ 26 , 27 ], isolated word recognition [ 28 ], small and large vocabulary and continuous speech recognition [ 29 , 30 ], as well as in disordered speech processing [ 7 , 8 , 12 , 13 , 31 , 32 , 33 , 34 , 35 , 36 ].…”

Section: Introductionmentioning

confidence: 99%

Artificial Neural Networks Combined with the Principal Component Analysis for Non-Fluent Speech Recognition

Świetlicka

Kuniszyk-Jóźkowiak

Świetlicki

2022

Sensors

View full text Add to dashboard Cite

The presented paper introduces principal component analysis application for dimensionality reduction of variables describing speech signal and applicability of obtained results for the disturbed and fluent speech recognition process. A set of fluent speech signals and three speech disturbances—blocks before words starting with plosives, syllable repetitions, and sound-initial prolongations—was transformed using principal component analysis. The result was a model containing four principal components describing analysed utterances. Distances between standardised original variables and elements of the observation matrix in a new system of coordinates were calculated and then applied in the recognition process. As a classifying algorithm, the multilayer perceptron network was used. Achieved results were compared with outcomes from previous experiments where speech samples were parameterised with the Kohonen network application. The classifying network achieved overall accuracy at 76% (from 50% to 91%, depending on the dysfluency type).

show abstract

Generative adversarial networks for speech processing: A review

Cited by 47 publications

References 35 publications

Modeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no access to speech data

Modeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no access to speech data

Automatic Speech Recognition Method Based on Deep Learning Approaches for Uzbek Language

Artificial Neural Networks Combined with the Principal Component Analysis for Non-Fluent Speech Recognition

Contact Info

Product

Resources

About