Confidence scoring based on backward language models

Abstract

In this paper we introduce the backward N-gram language model (LM) scores as a confidence measure in large vocabulary continuous speech recognition.Contrary to a forward N-gram LM, in which the probability of a word is dependent on the preceding words, a word in a backward N-gram LM is predicted based on the following words only. So the backward LM is a model for sentences read from the end to the beginning.We show on the benchmark 20k word Wall Street Journal recognition task that the backward LM scores contain information for the confidence measure that is complementary to the information in forward LM scores. The normalised cross entropy metric for confidence measures increases significantly from 18.5% to 23.3% when backward LM scores are added to a confidence measure which includes the forward LM scores.

Information Density and Quality Estimation Features as Translationese Indicators for Human Translation Classification

“…These features are extracted the same way as the suprisal features, but based on language models trained on sentence-level reversed text. The backward language model features are popular in translation quality estimation studies and show interesting results (Duchateau et al, 2002;Rubino et al, 2013b).…”

Linguistically Motivated Statistical Machine Translation

“…Sánchez and Benedí (2006) use a stochastic BTG to obtain bilingual phrases for phrase-based SMT. Duchateau et al (2002) use the score estimated by a backward language model in a post-processing step as a confidence measure to detect wrongly recognized words in speech recognition. Duchateau et al (2002) use the score estimated by a backward language model in a post-processing step as a confidence measure to detect wrongly recognized words in speech recognition.…”

Backward and trigger-based language models for statistical machine translation

“…Since the context ‘history’ in the backward language model is actually the future words to be generated, the backward language model is normally used in a post-processing where all words have already been generated or in a scenario where sentences are reversed. Duchateau, Demuynck and Wambacq (2002) use the backward language model score as a confidence measure to detect wrongly recognized words in speech recognition. Finch and Sumita (2009) use the backward language model in their reverse translation decoder where source sentences are reversed.…”