Tiny Transducer: A Highly-Efficient Speech Recognition Model on Edge Devices

Zhang, Yuekai; Sun, Sining; Ma, Long

doi:10.1109/icassp39728.2021.9413854

Cited by 16 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This architecture uses an extra token called a blank token (). During model testing, the probability of a blank token is usually quite large [13]. This led to an increase in deletion error where output missed many tokens.…”

Section: Blank Label Re-weightingmentioning

confidence: 99%

See 1 more Smart Citation

ASR - VLSP 2021: Automatic Speech Recognition with Blank Label Re-weighting

Thang

Son²,

Linh³

et al. 2022

JCSCE

View full text Add to dashboard Cite

End-to-end models have significant potential in most languages and recently proved the robustness in ASR tasks. Many robust architectures are proposed, and among many techniques, Recurrent Neural Network - Transducer (RNN-T) shows remarkable success. However, with background noise or reverb in spontaneous speech, this architecture generally suffers from high deletion error problems. For this reason, we propose the blank label re-weighting technique to improve the state-of-the-art Conformer transducer model. Our proposed system adopts the Stochastic Weight Averaging approach, stabilizing the training process. Our work achieved the first rank with a 4.17% of word error rate in Task 2 of the VLSP 2021 Competition.

show abstract

Section: Blank Label Re-weightingmentioning

confidence: 99%

“…To deal with this problem, we refer to the blank label re-weighting approach [13]. This original article proposed reducing the probability of a blank token.…”

Section: Blank Label Re-weightingmentioning

confidence: 99%

ASR - VLSP 2021: Automatic Speech Recognition with Blank Label Re-weighting

Thang

Son²,

Linh³

et al. 2022

JCSCE

View full text Add to dashboard Cite

show abstract

“…In the last few years there has been an increasing interest both in academia and in the industry to find ways to move the computation effort of deep learning inference from centralized clouds and servers closer to the network edge and to endpoint devices (mobiles, smart cameras, etc.). For widely deployed deep learning applications, such as speech recognition [1], performing inference on the endpoint device reduces network bandwidth usage, latency, and need for exposing potentially sensitive data to the network. On the other hand, deep learning inference requires the endpoint device to perform a significant amount of computations, which is potentially not feasible for very low-end mobile devices.…”

Section: Introductionmentioning

confidence: 99%

Edge-PRUNE: Flexible Distributed Deep Learning Inference

Boutellier¹,

Tan²,

Nurmi³

2022

Preprint

View full text Add to dashboard Cite

Collaborative deep learning inference between lowresource endpoint devices and edge servers has received significant research interest in the last few years. Such computation partitioning can help reducing endpoint device energy consumption and improve latency, but equally importantly also contributes to privacy-preserving of sensitive data. This paper describes Edge-PRUNE, a flexible but light-weight computation framework for distributing machine learning inference between edge servers and one or more client devices. Compared to previous approaches, Edge-PRUNE is based on a formal dataflow computing model, and is agnostic towards machine learning training frameworks, offering at the same time wide support for leveraging deep learning accelerators such as embedded GPUs. The experimental section of the paper demonstrates the use and performance of Edge-PRUNE by image classification and object tracking applications on two heterogeneous endpoint devices and an edge server, over wireless and physical connections. Endpoint device inference time for SSD-Mobilenet based object tracking, for example, is accelerated 5.8× by collaborative inference.

show abstract

“…In recent years, end-to-end (E2E) automatic speech recognition (ASR) technology has made great progress with its simplified architecture and competitive performance. Transducer [1,2,3,4,5,6] and attention based encoder-decoder (AED) [7,8,9] are two popular E2E frameworks. For both kinds of models, the encoder is crucial for good performance.…”

Section: Introductionmentioning

confidence: 99%

Efficient Conformer with Prob-Sparse Attention Mechanism for End-to-End Speech Recognition

et al. 2021

Self Cite

View full text Add to dashboard Cite

End-to-end models are favored in automatic speech recognition (ASR) because of their simplified system structure and superior performance. Among these models, Transformer and Conformer have achieved state-of-the-art recognition accuracy in which self-attention plays a vital role in capturing important global information. However, the time and memory complexity of self-attention increases squarely with the length of the sentence. In this paper, a prob-sparse self-attention mechanism is introduced into Conformer to sparse the computing process of self-attention in order to accelerate inference speed and reduce space consumption. Specifically, we adopt a Kullback-Leibler divergence based sparsity measurement for each query to decide whether we compute the attention function on this query. By using the prob-sparse attention mechanism, we achieve impressively 8% to 45% inference speed-up and 15% to 45% memory usage reduction of the self-attention module of Conformer Transducer while maintaining the same level of error rate.

show abstract

Tiny Transducer: A Highly-Efficient Speech Recognition Model on Edge Devices

Cited by 16 publications

References 23 publications

ASR - VLSP 2021: Automatic Speech Recognition with Blank Label Re-weighting

ASR - VLSP 2021: Automatic Speech Recognition with Blank Label Re-weighting

Edge-PRUNE: Flexible Distributed Deep Learning Inference

Efficient Conformer with Prob-Sparse Attention Mechanism for End-to-End Speech Recognition

Contact Info

Product

Resources

About