Transformer ASR with Contextual Block Processing

Tsunoo, Emiru; Kashiwagi, Yosuke; Kumakura, Toshiyuki; Watanabe, Shinji

doi:10.1109/asru46091.2019.9003749

Cited by 52 publications

(42 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors of [23] also reported robust effects on the LibriSpeech benchmark with transformers within the hybrid system. Several works [24][25][26], as well as transformer transducers [9,27], have demonstrated the effectiveness of transformer for online speech recognition. The authors of [28] also employed an unsupervised pre-training method to enhance the transformer for ASR.…”

Section: Existing Work On Transformers For Asrmentioning

confidence: 99%

A Bidirectional Context Embedding Transformer for Automatic Speech Recognition

et al. 2022

View full text Add to dashboard Cite

Transformers have become popular in building end-to-end automatic speech recognition (ASR) systems. However, transformer ASR systems are usually trained to give output sequences in the left-to-right order, disregarding the right-to-left context. Currently, the existing transformer-based ASR systems that employ two decoders for bidirectional decoding are complex in terms of computation and optimization. The existing ASR transformer with a single decoder for bidirectional decoding requires extra methods (such as a self-mask) to resolve the problem of information leakage in the attention mechanism This paper explores different options for the development of a speech transformer that utilizes a single decoder equipped with bidirectional context embedding (BCE) for bidirectional decoding. The decoding direction, which is set up at the input level, enables the model to attend to different directional contexts without extra decoders and also alleviates any information leakage. The effectiveness of this method was verified with a bidirectional beam search method that generates bidirectional output sequences and determines the best hypothesis according to the output score. We achieved a word error rate (WER) of 7.65%/18.97% on the clean/other LibriSpeech test set, outperforming the left-to-right decoding style in our work by 3.17%/3.47%. The results are also close to, or better than, other state-of-the-art end-to-end models.

show abstract

Section: Existing Work On Transformers For Asrmentioning

confidence: 99%

A Bidirectional Context Embedding Transformer for Automatic Speech Recognition

et al. 2022

View full text Add to dashboard Cite

show abstract

“…However, the global channel, speaker, and linguistic context are also important for local phoneme classification. Therefore, a context inheritance mechanism for block processing was proposed in [9,10] by introducing an additional context embedding vector in the encoder. Thus, the encoder sequentially computes encoded features h1:T b from the currently given b block input x1:T b .…”

Section: Streaming Encoder-decoder Asrmentioning

confidence: 99%

“…For interactive use cases in particular, streaming style inference is essential; thus, several approaches have been discovered for both the encoder-decoder (Enc-Dec) [1,2,3] and transducer models [4,5]. Blockwise processing can be easily introduced to the encoders of both models [6,7,8,9,10]. Although transducers are efficient for streaming ASR owing to framesynchronous decoding, they are less accurate than Enc-Dec [11] and Enc-Dec can be used additionally to achieve higher performance [12].…”

Section: Introductionmentioning

confidence: 99%

Run-and-back stitch search: novel block synchronous decoding for streaming encoder-decoder ASR

Tsunoo¹,

Narisetty²,

Hentschel³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

A streaming style inference of encoder-decoder automatic speech recognition (ASR) systems is important for reducing latency, which is essential for interactive use cases. To this end, we propose a novel blockwise synchronous decoding algorithm with a hybrid approach that combines endpoint prediction and endpoint post-determination. In the endpoint prediction, we compute the expectation of the number of tokens that are yet to be emitted in the encoder features of the current blocks using the CTC posterior. Based on the expectation value, the decoder predicts the endpoint to realize continuous block synchronization, as a running stitch. Meanwhile, endpoint post-determination probabilistically detects backward jump of the source-target attention, which is caused by the misprediction of endpoints. Then it resumes decoding by discarding those hypotheses, as back stitch. We combine these methods into a hybrid approach, namely run-and-back stitch search, which reduces the computational cost and latency. Evaluations of various ASR tasks show the efficiency of our proposed decoding algorithm, which achieves a latency reduction, for instance in the Librispeech test set from 1487 ms to 821 ms at the 90th percentile, while maintaining a high recognition accuracy.

show abstract

“…( 1), the attention is computed upon the full sequence of encoder and/or decoder states as required by the softmax function, which poses a big challenge for online recognition. To stream the Transformer ASR system, chunk-hopping based strategies [8,20,21,22] have been applied on the encoder side, where the input utterance is spliced into overlapping chunks and the chunks are chronologically fed to the SAE. Thus, the latency of the online encoder is subject to the chunk size.…”

Section: Transformer-based Online Asr Systemmentioning

confidence: 99%

Head-Synchronous Decoding for Transformer-Based Streaming ASR

Zorilă

Doddipatla

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Online Transformer-based automatic speech recognition (ASR) systems have been extensively studied due to the increasing demand for streaming applications. Recently proposed Decoder-end Adaptive Computation Steps (DACS) algorithm for online Transformer ASR was shown to achieve state-of-the-art performance and outperform other existing methods. However, like any other online approach, the DACS-based attention heads in each of the Transformer decoder layers operate independently (or asynchronously) and lead to diverged attending positions. Since DACS employs a truncation threshold to determine the halting position, some of the attention weights are cut off untimely and might impact the stability and precision of decoding. To overcome these issues, here we propose a head-synchronous (HS) version of the DACS algorithm, where the boundary of attention is jointly detected by all the DACS heads in each decoder layer. ASR experiments on Wall Street Journal (WSJ), AIShell-1 and Librispeech show that the proposed method consistently outperforms vanilla DACS and achieves state-of-the-art performance. We will also demonstrate that HS-DACS has reduced decoding cost when compared to vanilla DACS.

show abstract

Transformer ASR with Contextual Block Processing

Cited by 52 publications

References 35 publications

A Bidirectional Context Embedding Transformer for Automatic Speech Recognition

A Bidirectional Context Embedding Transformer for Automatic Speech Recognition

Run-and-back stitch search: novel block synchronous decoding for streaming encoder-decoder ASR

Head-Synchronous Decoding for Transformer-Based Streaming ASR

Contact Info

Product

Resources

About