Phoneme Based Neural Transducer for Large Vocabulary Speech Recognition

Zhou, Wei; Berger, Simon; Schlüter, Ralf; Ney, Hermann

doi:10.1109/icassp39728.2021.9413648

Cited by 21 publications

(14 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This corresponds to time-synchronous transducer models [6,7,10], where the RNN-T vertical transition is replaced with a diagonal transition, i.e. u = t and U = T .…”

Section: Special Case: Strict Monotonicitymentioning

confidence: 99%

“…One setup of our experiments is done on the 2nd release of the TED-LIUM corpus (TED-LIUM-v2) [22]. We use the same phoneme-based transducer model from [10], which has the strict monotonicity constraint as described in Section 2.5. Additionally, the model assumes a first-order dependency which still fits in the equivalence transformation shown in Section 2.3.…”

Section: Phoneme-based Transducer On Ted-lium-v2mentioning

confidence: 99%

“…( 10) for time-synchronous and label-synchronous search, respectively. The same 4-gram wordlevel LM and scale λ as in [10] are used for all recognition.…”

Section: Phoneme-based Transducer On Ted-lium-v2mentioning

confidence: 99%

“…Most acoustic models in speech recognition can be categorized into being defined on the level of each time frame or directly on the level of labels and/or segments. The hybrid neural network (NN) -hidden Markov model (HMM) [1,2] and extensions [3], CTC [4] and their generalized transducer variants [5][6][7][8][9][10] are all defined on the time-frame level. The attentionbased encoder-decoder model [11][12][13][14][15] and in general segmental models [16][17][18][19][20] are defined on a label and/or segment level.…”

Section: Introductionmentioning

confidence: 99%

“…This equivalence implies that we can use both label-synchronous or time-synchronous beam search decoding for either case. We provide initial experiments comparing label-synchronous and time-synchronous beam search decoding for the transducer models introduced in [9,10].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Equivalence of Segmental and Neural Transducer Modeling: A Proof of Concept

Zhou¹,

Zeyer²,

Merboldt³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

With the advent of direct models in automatic speech recognition (ASR), the formerly prevalent frame-wise acoustic modeling based on hidden Markov models (HMM) diversified into a number of modeling architectures like encoder-decoder attention models, transducer models and segmental models (direct HMM). While transducer models stay with a frame-level model definition, segmental models are defined on the level of label segments, directly. While (soft-)attention-based models avoid explicit alignment, transducer and segmental approach internally do model alignment, either by segment hypotheses or, more implicitly, by emitting so-called blank symbols. In this work, we prove that the widely used class of RNN-Transducer models and segmental models (direct HMM) are equivalent and therefore show equal modeling power. It is shown that blank probabilities translate into segment length probabilities and vice versa. In addition, we provide initial experiments investigating decoding and beam-pruning, comparing time-synchronous and label-/segment-synchronous search strategies and their properties using the same underlying model.

show abstract

“…This corresponds to time-synchronous transducer models [6,7,10], where the RNN-T vertical transition is replaced with a diagonal transition, i.e. u = t and U = T .…”

Section: Special Case: Strict Monotonicitymentioning

confidence: 99%

Section: Phoneme-based Transducer On Ted-lium-v2mentioning

confidence: 99%

“…( 10) for time-synchronous and label-synchronous search, respectively. The same 4-gram wordlevel LM and scale λ as in [10] are used for all recognition.…”

Section: Phoneme-based Transducer On Ted-lium-v2mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Equivalence of Segmental and Neural Transducer Modeling: A Proof of Concept

Zhou¹,

Zeyer²,

Merboldt³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Developing phoneme‐based lip‐reading sentences system for silent speech recognition

El-Bialy

Chen

Fenghour

et al. 2022

CAAI Trans on Intel Tech

View full text Add to dashboard Cite

Lip-reading is a process of interpreting speech by visually analysing lip movements. Recent research in this area has shifted from simple word recognition to lip-reading sentences in the wild. This paper attempts to use phonemes as a classification schema for lip-reading sentences to explore an alternative schema and to enhance system performance. Different classification schemas have been investigated, including characterbased and visemes-based schemas. The visual front-end model of the system consists of a Spatial-Temporal (3D) convolution followed by a 2D ResNet. Transformers utilise multi-headed attention for phoneme recognition models. For the language model, a Recurrent Neural Network is used. The performance of the proposed system has been testified with the BBC Lip Reading Sentences 2 (LRS2) benchmark dataset. Compared with the state-of-the-art approaches in lip-reading sentences, the proposed system has demonstrated an improved performance by a 10% lower word error rate on average under varying illumination ratios. K E Y W O R D S deep learning, deep neural networks, lip-reading, phoneme-based lip-reading, spatial-temporal convolution, transformers This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architectures

Rossenbach

Zeineldeen

Hilmes

et al. 2021

2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

Self Cite

View full text Add to dashboard Cite

Synthetic data generated by text-to-speech (TTS) systems can be used to improve automatic speech recognition (ASR) systems in low-resource or domain mismatch tasks. It has been shown that TTS-generated outputs still do not have the same qualities as real data. In this work we focus on the temporal structure of synthetic data and its relation to ASR training. By using a novel oracle setup we show how much the degradation of synthetic data quality is influenced by duration modeling in non-autoregressive (NAR) TTS. To get reference phoneme durations we use two common alignment methods, a hidden Markov Gaussian-mixture model (HMM-GMM) aligner and a neural connectionist temporal classification (CTC) aligner. Using a simple algorithm based on random walks we shift phoneme duration distributions of the TTS system closer to real durations, resulting in an improvement of an ASR system using synthetic data in a semi-supervised setting.

show abstract

Phoneme Based Neural Transducer for Large Vocabulary Speech Recognition

Cited by 21 publications

References 31 publications

Equivalence of Segmental and Neural Transducer Modeling: A Proof of Concept

Equivalence of Segmental and Neural Transducer Modeling: A Proof of Concept

Developing phoneme‐based lip‐reading sentences system for silent speech recognition

Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architectures

Contact Info

Product

Resources

About