A study of continuous space word and sentence representations applied to ASR error detection

Abstract

This paper presents a study of continuous word representations applied to automatic detection of speech recognition errors. A neural network architecture is proposed, which is well suited to handle continuous word representations, like word embeddings. We explore the use of several types of word representations: simple and combined linguistic embeddings, and acoustic ones associated to prosodic features, extracted from the audio signal. To compensate certain phenomena highlighted by the analysis of the error average span, we propose to model the errors at the sentence level through the use of sentence embeddings. An approach to build continuous sentence representations dedicated to ASR error detection is also proposed and compared to the Doc2vec approach. Experiments are performed on automatic transcriptions generated by the LIUM ASR system applied to the French ETAPE corpus. They show that the combination of linguistic embeddings, acoustic embeddings, prosodic features, and sentence embeddings in addition to more classical features yields very competitive results. Particularly, these results show the complementarity of acoustic embeddings and prosodic information, and show that the proposed sentence embeddings dedicated to ASR error detection achieve better results than generic sentence embeddings.

Keep the noise down: On the performance of automatic speech recognition of voice-recordings in web surveys

“…We developed a coding schema based on previous research on common ASR error types and sources (Errattahi et al 2019;Ghannay, Estève, and Camelin 2020). It distinguishes between error types (e.g., error in verbs, nouns) and error sources (e.g., background noise, pronunciation errors) and assessed whether the content/meaning of the response changed due to the transcription.…”

“…However, the transcription and the voice-recording might differ. The accuracy of ASR transcriptions might be low due to longer, shorter, missing, added text, or compound words (Errattahi et al 2019;Ghannay, Estève, and Camelin 2020). A common measure of ASR accuracy is the word error rate (WER) which is the number of transcription errors divided by the answer length (Kim et al 2019;Tancoigne et al 2022).…”

A Review About RNA–Protein-Binding Sites Prediction Based on Deep Learning

“…However, the GANs have difficulties in dealing with discrete data. In natural languages processing, the text sequences [62] are evaluated as the discrete tokens whose values are non-differentiable. Therefore, the optimization of GANs is challenging.…”