Exploring the Encoder Layers of Discriminative Autoencoders for LVCSR

Huang, Pin-Tuan; Lee, Hung-Shin; Wang, Syu-Siang; Chen, Kuan‐Yu; Wang, Hsin‐Min

doi:10.21437/interspeech.2019-1717

Cited by 8 publications

(4 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The sequence-to-sequence (S2S) approach [1] has achieved remarkable results in automatic speech recognition (ASR), in particular, large vocabulary continuous speech recognition (LVCSR) [2][3][4][5][6][7][8]. Unlike the conventional hybrid ASR, S2S requires neither lexicons, nor prerequisite models or decision trees.…”

Section: Introductionmentioning

confidence: 99%

Self-and-Mixed Attention Decoder with Deep Acoustic Structure for Transformer-Based LVCSR

Zhou¹,

Lee²,

Yılmaz³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

The Transformer has shown impressive performance in automatic speech recognition. It uses the encoder-decoder structure with self-attention to learn the relationship between the high-level representation of the source inputs and embedding of the target outputs. In this paper, we propose a novel decoder structure that features a self-and-mixed attention decoder (SMAD) with a deep acoustic structure (DAS) to improve the acoustic representation of Transformer-based LVCSR. Specifically, we introduce a self-attention mechanism to learn a multilayer deep acoustic structure for multiple levels of acoustic abstraction. We also design a mixed attention mechanism that learns the alignment between different levels of acoustic abstraction and its corresponding linguistic information simultaneously in a shared embedding space. The ASR experiments on Aishell-1 shown that the proposed structure achieves CERs of 4.8% on the dev set and 5.1% on the test set, which are the best results obtained on this task to the best of our knowledge.

show abstract

Section: Introductionmentioning

confidence: 99%

Self-and-Mixed Attention Decoder with Deep Acoustic Structure for Transformer-Based LVCSR

Zhou¹,

Lee²,

Yılmaz³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

show abstract

“…where α and β are weighting factors. In previous studies in [29,30,31], only the CE and MSE were jointly optimized in AM training. In our experiments, we set α to 5 as in most previous studies and heuristically set β to 0.2.…”

Section: Final Objective Functionmentioning

confidence: 99%

Speech-enhanced and Noise-aware Networks for Robust Speech Recognition

Lee¹,

Chen²,

Yu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Compensation for channel mismatch and noise interference is essential for robust automatic speech recognition. Enhanced speech has been introduced into the multi-condition training of acoustic models to improve their generalization ability. In this paper, a noise-aware training framework based on two cascaded neural structures is proposed to jointly optimize speech enhancement and speech recognition. The feature enhancement module is composed of a multi-task autoencoder, where noisy speech is decomposed into clean speech and noise. By concatenating its enhanced, noise-aware, and noisy features for each frame, the acoustic-modeling module maps each featureaugmented frame into a triphone state by optimizing the latticefree maximum mutual information and cross entropy between the predicted and actual state sequences. On top of the factorized time delay neural network (TDNN-F) and its convolutional variant (CNN-TDNNF), both with SpecAug, the two proposed systems achieve word error rate (WER) of 3.90% and 3.55%, respectively, on the Aurora-4 task. Compared with the best existing systems that use bigram and trigram language models for decoding, the proposed CNN-TDNNF-based system achieves a relative WER reduction of 15.20% and 33.53%, respectively. In addition, the proposed CNN-TDNNF-based system also outperforms the baseline CNN-TDNNF system on the AMI task.

show abstract

“…To import the advantage of unsupervised learning in [15], [16] into our acoustic modeling, the DNN model for generating the emission probabilities of the output labels can be regarded as the encoder, and the decoder is used to reconstruct the acoustic features, as shown in Figure 2. The output of the penultimate layer of the encoder is supposed to be a pure phoneme-related vector without any information irrelevant to the phonetic content.…”

Section: Proposed Model a Filtering Mechanism For Acoustic Modelingmentioning

confidence: 99%

“…In [15], [16], discriminative autoencoder-based (DcAE) acoustic modeling was proposed to separate the acoustic feature into the components of phoneme, speaker and environmental noise. Such model-space innovation takes a great advantage of unsupervised learning to extract pure phonetic components from the acoustic feature to better recognize speech.…”

Section: Introductionmentioning

confidence: 99%

Filter-based Discriminative Autoencoders for Children Speech Recognition

Tai¹,

Lee²,

Yu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Children speech recognition is indispensable but challenging due to the diversity of children's speech. In this paper, we propose a filter-based discriminative autoencoder for acoustic modeling. To filter out the influence of various speaker types and pitches, auxiliary information of the speaker and pitch features is input into the encoder together with the acoustic features to generate phonetic embeddings. In the training phase, the decoder uses the auxiliary information and the phonetic embedding extracted by the encoder to reconstruct the input acoustic features. The autoencoder is trained by simultaneously minimizing the ASR loss and feature reconstruction error. The framework can make the phonetic embedding purer, resulting in more accurate senone (triphone-state) scores. Evaluated on the test set of the CMU Kids corpus, our system achieves a 7.8% relative WER reduction compared to the baseline system. In the domain adaptation experiment, our system also outperforms the baseline system on the British-accent PF-STAR task.

show abstract

Exploring the Encoder Layers of Discriminative Autoencoders for LVCSR

Cited by 8 publications

References 29 publications

Self-and-Mixed Attention Decoder with Deep Acoustic Structure for Transformer-Based LVCSR

Self-and-Mixed Attention Decoder with Deep Acoustic Structure for Transformer-Based LVCSR

Speech-enhanced and Noise-aware Networks for Robust Speech Recognition

Filter-based Discriminative Autoencoders for Children Speech Recognition

Contact Info

Product

Resources

About