CRF-based Single-stage Acoustic Modeling with CTC Topology

Xiang, Hongyu; Ou, Zhijian

doi:10.1109/icassp.2019.8682256

Cited by 27 publications

(39 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hu et al [17] used several improved Darts based methods to find efficient context offset and bottleneck dimension for acoustic model. Zheng et al [18] proposed straight-through gradient search beyond SNAS [24] and ProxylessNAS [25] and trained acoustic model with CRF-CTC [26], but the search space was still limited to receptive field and dimension of TDNN-F only. Kim et al [19] proposed a relatively complex search space: evolved Transformer, split the encoder and decoder of Transformer into left and right branch and iterated the population with progressive dynamic hurdles (PDH) [4].…”

Section: Nas In Speech Fieldmentioning

confidence: 99%

Darts-Conformer: Towards Efficient Gradient-Based Neural Architecture Search For End-to-End ASR

Shi¹,

Zhou²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Neural architecture search (NAS) has been successfully applied to tasks like image classification and language modeling for finding efficient high-performance network architectures. In ASR field especially end-to-end ASR, the related research is still in its infancy. In this work, we focus on applying NAS on the most popular manually designed model: Conformer, and then propose an efficient ASR model searching method that benefits from the natural advantage of differentiable architecture search (Darts) in reducing computational overheads. We fuse Darts mutator and Conformer blocks to form a complete search space, within which a modified architecture called Darts-Conformer cell is found automatically. The entire searching process on AISHELL-1 dataset costs only 0.7 GPU days. Replacing the Conformer encoder by stacking searched cell, we get an end-to-end ASR model (named as Darts-Conformner) that outperforms the Conformer baseline by 4.7% on the opensource AISHELL-1 dataset. Besides, we verify the transferability of the architecture searched on a small dataset to a larger 2k-hour dataset. To the best of our knowledge, this is the first successful attempt to apply gradient-based architecture search in the attention-based encoder-decoder ASR model.

show abstract

Section: Nas In Speech Fieldmentioning

confidence: 99%

Darts-Conformer: Towards Efficient Gradient-Based Neural Architecture Search For End-to-End ASR

Shi¹,

Zhou²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…There are recent attempts to use end-toend ASR models such as CTC with monophones [8,25] or end-toend LF-MMI with biphones [28,27] for multilingual and crosslingual recognition. Remarkably, the end-to-end CTC-CRF model, which is defined by a CRF (conditional random field) with CTC topology, has been shown to perform significantly better than CTC [20,21]. Moreover, mono-phone CTC-CRF performs comparably to bi-phone end-to-end LF-MMI [28] and avoids context-dependent modeling with a simpler pipeline, which is particularly attractive for multilingual and crosslingual speech recognition.…”

Section: Related Workmentioning

confidence: 99%

“…In this section, we briefly explain the CTC-CRF based framework [20,21] to use phone embeddings for ASR. Consider discriminative training with the objective to maximize the conditional likelihood [20]:…”

Section: Ctc-crf Based Asrmentioning

confidence: 99%

“…In contrast, by incorporating log p(l) into the potential function in CTC-CRF, this drawback is naturally avoided. It has been shown that CTC-CRF outperforms regular CTC consistently on a wide range of benchmarks, and is on par with other state-of-the-art end-to-end models [20,21,30]. Moreover, CTC-CRF enjoys data-efficiency in training and works well with monophones [21], which are favorable for low-resourced multilingual and crosslingual speech recognition.…”

Section: Ctc-crf Based Asrmentioning

confidence: 99%

“…Next, we conduct bottom-up calculation on an acoustic DNN, viewed as a cascade of acoustic feature extractors. Finally, the extracted acoustic features and the phone embeddings are joined to calculate the phone (posterior) probabilities, which can be further used to calculate the CTC loss [19] or the CTC-CRF loss [20,21]. This completes the definition of a multilingual acoustic model, which involves the Joining of Acoustics and Phonology and is thus called the JoinAP method.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multilingual and crosslingual speech recognition using phonological-vector based phone embeddings

Zhu¹,

An²,

Zheng³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The use of phonological features (PFs) potentially allows language-specific phones to remain linked in training, which is highly desirable for information sharing for multilingual and crosslingual speech recognition methods for low-resourced languages. A drawback suffered by previous methods in using phonological features is that the acoustic-to-PF extraction in a bottom-up way is itself difficult. In this paper, we propose to join phonology driven phone embedding (top-down) and deep neural network (DNN) based acoustic feature extraction (bottom-up) to calculate phone probabilities. The new method is called JoinAP (Joining of Acoustics and Phonology). Remarkably, no inversion from acoustics to phonological features is required for speech recognition. For each phone in the IPA (International Phonetic Alphabet) table, we encode its phonological features to a phonological-vector, and then apply linear or nonlinear transformation of the phonological-vector to obtain the phone embedding. A series of multilingual and crosslingual (both zero-shot and few-shot) speech recognition experiments are conducted on the CommonVoice dataset (German, French, Spanish and Italian) and the AISHLL-1 dataset (Mandarin), and demonstrate the superiority of JoinAP with nonlinear phone embeddings over both JoinAP with linear phone embeddings and the traditional method with flat phone embeddings.

show abstract