Improved statistical models for SMT-based speaking style transformation

Abstract

Automatic speech recognition (ASR) results contain not only ASR errors, but also disfluencies and colloquial expressions that must be corrected to create readable transcripts. We take the approach of statistical machine translation (SMT) to "translate" from ASR results into transcript-style text. We introduce two novel modeling techniques in this framework: a context-dependent translation model, which allows for usage of context to accurately model translation probabilities, and log-linear interpolation of conditional and joint probabilities, which allows for frequently observed translation patterns to be given higher priority. The system is implemented using weighted finite state transducers (WFST). On an evaluation using ASR results and manual transcripts of meetings of the Japanese Diet (national congress), the proposed methods showed a significant increase in accuracy over traditional modeling techniques.

Improving the Readability of ASR Results for Lectures Using Multiple Hypotheses and Sentence-Level Knowledge

“…In the future, we will address taking a more sophisticated transformation model to deal with the correction of colloquial expressions like [10], which are difficult to consider in an unsupervised manner. In addition, if the construction of a confusion network is problematic, for example, in real-time applications, it is better to use lattices instead of confusion networks to represent multiple hypotheses.…”

“…(6) plays an important role in improving the readability of ASR results, because our method assumes that there are no parallel data to obtain transformation rules between raw and cleaned transcripts, unlike [10], and only uses a few heuristic rules for the transformation.…”

“…We consider multiple hypotheses from a decoder by a max operator while traditional methods based on statistical methods consider only the single-best hypothesis and thus use only P(w|s) [7]- [10].…”

“…Operations for correcting transcripts include removing filled pauses and repetition, converting from a spoken style to a written style, inserting punctuation marks such as commas and periods, and discourse markers such as for paragraphs. In this context, there is currently extensive research on paraphrasing and correcting recognition results [7]- [10]. Shitaoka et al formulated the problem as a kind of machine translation and applied a statistical method to transform spoken language to written language [7].…”

“…Hori et al used weighted finite state transducers (WFSTs) for the same purpose by representing each component as a WFST [8]. Neibig et al also used WFSTs in which their method was based on the WFST-based log-linear framework [9], [10].…”