Log Likelihood Ratio Based Annotation Verification of a Norwegian Speech Synthesis Database

Amdal, Ingunn; Johnsen, Magne Hallstein; Svendsen, Torbjørn

doi:10.1109/norsig.2006.275206

Cited by 2 publications

(2 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The set contains approximately 2600 sentence-like segments, all with manual reference texts from their respective corpora. We use the free speech test set of NB Tale [19], which contains spontaneous narrative monologues, and a part of the test set of Rundkast [20], which includes broadcast radio news shows. Lastly, we take the test set from the Norwegian Parliamentary Speech Corpus (NPSC) [21].…”

Section: Survey On Human Perception Of Asr Transcription Qualitymentioning

confidence: 99%

Perceptual and Task-Oriented Assessment of a Semantic Metric for ASR Evaluation

Rugayan¹,

Salvi²,

Svendsen³

2023

Interspeech 2023

View full text Add to dashboard Cite

Automatic speech recognition (ASR) systems have become a vital part of our everyday lives through their many applications. However, as much as we have developed in this regard, our most common evaluation method for ASR systems still remains to be word error rate (WER). WER does not give information on the severity of errors, which strongly impacts practical performance. As such, we examine a semantic-based metric called Aligned Semantic Distance (ASD) against WER and demonstrate its advantage over WER in two facets. First, we conduct a survey asking participants to score reference text and ASR transcription pairs. We perform a correlation analysis and show that ASD is more correlated to the human evaluation scores compared to WER. We also explore the feasibility of predicting human perception using ASD. Second, we demonstrate that ASD is more effective than WER as an indicator of performance on downstream NLP tasks such as named entity recognition and sentiment classification.

show abstract

Section: Survey On Human Perception Of Asr Transcription Qualitymentioning

confidence: 99%

Perceptual and Task-Oriented Assessment of a Semantic Metric for ASR Evaluation

Rugayan¹,

Salvi²,

Svendsen³

2023

Interspeech 2023

View full text Add to dashboard Cite

show abstract

“…In this paper, we proposed to locate the heteronym annotation errors from the annotation verification view, and use log likelihood ratio [2] of various features to locate the heteronym annotation errors in a Mandarin speech synthesis database. For that Mandarin is a tonal language, and heteronym can be pronounced differently either in the phone itself or just in tone, we divide the heteronyms into 2 categories: class A is the heteronym pronounced differently just in tone, and class B is the ones pronounced different in phone and either the same or different in tone.…”

Section: Introductionmentioning

confidence: 99%

Heteronym Verification for Mandarin Speech Synthesis

Ling

Wei

et al. 2008

2008 6th International Symposium on Chinese Spoken Language Processing

View full text Add to dashboard Cite

Accurate phonetic transcription of speech corpus is critical to high quality speech synthesis. In Mandarin text-to-speech (MTTS) system, one major problem of automatically labeling the database is the heteronym annotation. Because in Mandarin, there are some single-character words or multi-character words have more than one pronunciation. In this paper, a heteronym annotation verification method for MTTS database labeling is proposed. By training contextual dependent HMMs and calculating the log likelihood ratio, each heteronym in the database is assigned a confidence score and those below the threshold are selected for manual inspecting. We divide heteronyms in Mandarin into two categories and different features are used for each category. The result of our experiment on an artificial test set has shown that we can achieve EER (equal error rate) of 7.9% and 11.9% for these two categories. Further test on an actual database which contains a total of 36098 heteronyms has shown that the proposed method can find 89 of all 123 annotation errors by only inspecting 639 polyphones.

show abstract

Log Likelihood Ratio Based Annotation Verification of a Norwegian Speech Synthesis Database

Cited by 2 publications

References 4 publications

Perceptual and Task-Oriented Assessment of a Semantic Metric for ASR Evaluation

Perceptual and Task-Oriented Assessment of a Semantic Metric for ASR Evaluation

Heteronym Verification for Mandarin Speech Synthesis

Contact Info

Product

Resources

About