Lexicon-based vs. Lexicon-free ASR for Norwegian Parliament Speech Transcription

Nouza, Jan; Červa, Petr; Žďánský, Jindřich

doi:10.1007/978-3-031-16270-1_33

Cited by 3 publications

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We generate a set of ASR transcriptions using a strong baseline end-to-end ASR system [16] with hybrid connectionist temporal classification (CTC) [17] and attention-based encoder-1 https://huggingface.co/ bert-base-multilingual-cased decoder (AED) [18] architecture. The set contains approximately 2600 sentence-like segments, all with manual reference texts from their respective corpora.…”

Section: Survey On Human Perception Of Asr Transcription Qualitymentioning

confidence: 99%

Perceptual and Task-Oriented Assessment of a Semantic Metric for ASR Evaluation

Rugayan¹,

Salvi²,

Svendsen³

2023

Interspeech 2023

View full text Add to dashboard Cite

Automatic speech recognition (ASR) systems have become a vital part of our everyday lives through their many applications. However, as much as we have developed in this regard, our most common evaluation method for ASR systems still remains to be word error rate (WER). WER does not give information on the severity of errors, which strongly impacts practical performance. As such, we examine a semantic-based metric called Aligned Semantic Distance (ASD) against WER and demonstrate its advantage over WER in two facets. First, we conduct a survey asking participants to score reference text and ASR transcription pairs. We perform a correlation analysis and show that ASD is more correlated to the human evaluation scores compared to WER. We also explore the feasibility of predicting human perception using ASD. Second, we demonstrate that ASD is more effective than WER as an indicator of performance on downstream NLP tasks such as named entity recognition and sentiment classification.

show abstract

Section: Survey On Human Perception Of Asr Transcription Qualitymentioning

confidence: 99%