Online Speech Recognition Using Multichannel Parallel Acoustic Score Computation and Deep Neural Network (DNN)- Based Voice-Activity Detector

Abstract

This paper aims to design an online, low-latency, and high-performance speech recognition system using a bidirectional long short-term memory (BLSTM) acoustic model. To achieve this, we adopt a server-client model and a context-sensitive-chunk-based approach. The speech recognition server manages a main thread and a decoder thread for each client and one worker thread. The main thread communicates with the connected client, extracts speech features, and buffers the features. The decoder thread performs speech recognition, including the proposed multichannel parallel acoustic score computation of a BLSTM acoustic model, the proposed deep neural network-based voice activity detector, and Viterbi decoding. The proposed acoustic score computation method estimates the acoustic scores of a context-sensitive-chunk BLSTM acoustic model for the batched speech features from concurrent clients, using the worker thread. The proposed deep neural network-based voice activity detector detects short pauses in the utterance to reduce response latency, while the user utters long sentences. From the experiments of Korean speech recognition, the number of concurrent clients is increased from 22 to 44 using the proposed acoustic score computation. When combined with the frame skipping method, the number is further increased up to 59 clients with a small accuracy degradation. Moreover, the average user-perceived latency is reduced from 11.71 s to 3.09–5.41 s by using the proposed deep neural network-based voice activity detector.

Fast offline transformer‐based end‐to‐end automatic speech recognition for real‐world applications

“…For the batch processing of , the restricted range is defined as follows: The time‐restricted CTC‐prefix score can be computed in a small amount of the calculation. For a small but frequent calculation in a GPU device, memory transfer load between CPU and GPU can be a bottleneck [35]. Therefore, the calculation of the CTC‐prefix score is moved from the GPU device to a CPU device to reduce the GPU memory transfer load.…”

“…These methods suffer when the signal‐to‐noise ratio is dynamically changes. In our previous work [35], the outputs of acoustic model (AM) were successfully employed to detect voice activity in DNN‐HMM‐based ASR systems.…”

“…In a previous work [35], these probabilities were successfully used to compute probabilities of voice presence in input signal as follows.…”

“…To obtain a robust decision, the frame‐wise results are smoothened over multiple frames, as in Oh et al. [35].…”

Atomicity Violation in Multithreaded Applications and Its Detection in Static Code Analysis Process

“…Multithreading is one of the basic mechanisms for parallel computing, which is possible on one processor and on several processors [3,4]. It is used very often in utility applications (mentioned in the rest of the work), but also in scientific research [5]. Popular, commonly used open-source multithreaded applications are a very good source for research on undesirable phenomena in multithreaded applications.…”