Monolingual and crosslingual comparison of tandem features derived from articulatory and phone MLPS

Abstract

In recent years, the features derived from posteriors of a multilayer perceptron (MLP), known as tandem features, have proven to be very effective for automatic speech recognition. Most tandem features to date have relied on MLPs trained for phone classification. We recently showed on a relatively small data set that MLPs trained for articulatory feature classification can be equally effective. In this paper, we provide a similar comparison using MLPs trained on a much larger data set-2000 hours of English conversational telephone speech. We also explore how portable phone-and articulatory featurebased tandem features are in an entirely different language-Mandarin-without any retraining. We find that while the phone-based features perform slightly better in the matchedlanguage condition, they perform significantly better in the cross-language condition. Yet, in the cross-language condition, neither approach is as effective as the tandem features extracted from an MLP trained on a relatively small amount of in-domain data. Beyond feature concatenation, we also explore novel observation modeling schemes that allow for greater flexibility in combining the tandem and standard features at hidden Markov model (HMM) outputs.

Regularized Subspace Gaussian Mixture Models for Speech Recognition

“…Tandem features, based on phone posterior probability estimates, were originally proposed to improve monolingual speech recognition [11], but they have also proven effective in the cross-lingual setting. In this approach, multi-layer perceptrons (MLPs) trained using source language acoustic data of source language, are used to generate the MLP phone posterior features for the target language [12], [13], [14], [15]. As tandem acoustic features are not directly dependent on the lexicon, this approach is simple to apply.…”

Regularized subspace Gaussian mixture models for cross-lingual speech recognition

“…In this approach, multi-layer perceptrons (MLPs) trained using source language acoustic data of source language, are used to generate the MLP phone posterior features for the target language [12], [13], [14], [15]. As tandem acoustic features are not directly dependent on the lexicon, this approach is simple to apply.…”

HMM-based Approaches to Model Multichannel Information in Sign Language Inspired from Articulatory Features-based Speech Processing

“…One of the common approach to model AFs is to estimate these features using ANNs; transform them using tandem feature extraction technique; concatenate them with the acoustic feature; and model them with HMMs [14,15,18,19]. We can adopt a similar approach for SL processing where the features representing different channels of information are extracted, concatenated…”