Reducing Spelling Inconsistencies in Code-Switching ASR Using Contextualized CTC Loss

Abstract

Code-Switching (CS) remains a challenge for Automatic Speech Recognition (ASR), especially character-based models. With the combined choice of characters from multiple languages, the outcome from character-based models suffers from phoneme duplication, resulting in language-inconsistent spellings. We propose Contextualized Connectionist Temporal Classification (CCTC) loss to encourage spelling consistencies of a character-based nonautoregressive ASR which allows for faster inference. The model trained by CCTC loss is aware of contexts since it learns to predict both center and surrounding letters in a multi-task manner. In contrast to existing CTC-based approaches, CCTC loss does not require frame-level alignments, since the context ground truth is obtained from the model's estimated path. Compared to the same model trained with regular CTC loss, our method consistently improved the ASR performance on both CS and monolingual corpora.

A context-aware knowledge transferring strategy for CTC-based ASR

“…They are not only required to model the acoustic information from the speech signal but needed to generate a precise token sequence corresponding to the speech and the contextual coherence. In recent years, connectionist temporal classification (CTC)-based ASR systems [1] have attracted significant attention since they can achieve a much faster decoding speed in the non-autoregressive manner and obtain competitive or even better performance compared to the conventional auto-regressive models [2,3,4,5,6,7,8]. To be specific, a standard CTC-based ASR usually consists of a multi-layer Transformer-based acoustic encoder and a classification head based on some layers of simple feedforward neural network.…”

“…These methods mainly focus on easing the conditional independence assumption from a theoretical perspective. The contextualized CTC loss [6] is proposed to guild the model learn contextualized information by introducing extra prediction heads to predict surrounding tokens. Some studies aspire to improve CTC-based ASR via knowledge transferring from pre-trained language models [7].…”

Unlocking the power of co-collaboration: the impact of educator involvement in educational technology development

“…Sandberg (2022) stated that mastering written English is a complex process that varies for each student. This complexity requires students to map the phoneme-grapheme (sound-letter) relationships of regular or rule-following words and irregular or exceptional words by learning the formation of letter-sound connections, pronunciations, and word meanings, then retrieve the spelling from memory and perhaps code-switch, alternate between two or more languages or dialects, based upon native background (Arfé & Danzak, 2020;Ehri, 2014;Esposito et al, 2022;Naowarat et al, 2021). Failure to meet these requirements will result in spelling insufficiencies.…”