Efficient decoding with generative score-spaces using the expectation semiring

Abstract

State-of-the-art speech recognisers are usually based on hidden Markov models (HMMs). They model a hidden symbol sequence with a Markov process, with the observations independent given that sequence. These assumptions yield efficient algorithms, but limit the power of the model. An alternative model that allows a wide range of features, including word-and phone-level features, is a log-linear model. To handle, for example, word-level variable-length features, the original feature vectors must be segmented into words. Thus, decoding must find the optimal combination of segmentation of the utterance into words and word sequence. Features must therefore be extracted for each possible segment of audio. For many types of features, this becomes slow. In this paper, long-span features are derived from the likelihoods of word HMMs. Derivatives of the log-likelihoods, which break the Markov assumption, are appended. Previously, decoding with this model took cubic time in the length of the sequence, and longer for higher-order derivatives. This paper shows how to decode in quadratic time.

Structured Discriminative Models For Speech Recognition: An Overview

“…Alternatively, if GMMs are used, then derivative score spaces yields frame-level features; the derivative with respect to the component priors, for example, yields sparse GMM posterior features [44]. An interesting aspect of using structured generative models in this fashion is that feature extraction can be made efficient using an expectation semiring within the WFST framework [52]. Similar in spirit to the score-space paradigm are other methods that utilize detections of longer-term acoustic events.…”

“…[ [7], [19], [20], [35] SEGMENT-LEVEL SCORE SPACES [18], [44], [51], [52] SEGMENT-LEVEL MODEL FEATURES ( , ) ( , [17], [21], [43], [56]- [58] ONE OF THE INTERESTING ASPECTS OF SUPRASEGMENTAL FEATURES IS THAT THEY CAN BE EASILY COMBINED WITH GENERATIVE MODELS FOR CLASSIFICATION. which are combined with word and pronunciation feature functions.…”

A WFST Framework for Single-Pass Multi-Stream Decoding

“…For example, the lexicographic semiring by Shafran et al [2] is used for determinizing tagged word lattices. Also, van Dalen et al [3] report the use of expectation semiring to efficiently extract features.…”

Annotating large lattices with the exact word error

“…Many operations in speech recognition can be elegantly described in terms of finite-state automata [1,2,3,4]. However, optimisation algorithms do not always create the desired results.…”