Nonparametric Bayesian Double Articulation Analyzer for Direct Language Acquisition From Continuous Speech Signals

Abstract

Human infants can discover words directly from unsegmented speech signals without any explicitly labeled data. The main problem of this paper is to develop a computational model that can estimate language and acoustic models, and discover words directly from continuous human speech signals in an unsupervised manner. For this purpose, we propose an integrative generative model that combines a language model and an acoustic model into a single generative model called the "hierarchical Dirichlet process hidden language model" (HDP-HLM). The HDP-HLM is obtained by extending the hierarchical Dirichlet process hidden semi-Markov model (HDP-HSMM) proposed by Johnson et al. An inference procedure for the HDP-HLM is derived using the blocked Gibbs sampler originally proposed for the HDP-HSMM. This procedure enables the simultaneous and direct inference of language and acoustic models from continuous speech signals. Based on the HDP-HLM and its inference procedure, we develop a novel machine learning method called nonparametric Bayesian double articulation analyzer (NPB-DAA) that can directly acquire language and acoustic models from observed continuous speech signals. By assuming HDP-HLM as a generative model of observed time series data, and by inferring latent variables of the model, the method can analyze latent double articulation structure, i.e., hierarchically organized latent words and phonemes, of the data in an unsupervised manner. We also carried out two evaluation experiments using synthetic data and actual human continuous speech signals representing Japanese vowel sequences. In the word acquisition and phoneme categorization tasks, the NPB-DAA outperformed a conventional double articulation analyzer (DAA) and baseline automatic speech recognition system whose acoustic model was trained in a supervised manner. The main contributions of this paper are as follows: (1) We develop a probabilistic generative model that integrates language and acoustic models, i.e., HDP-HLM. (2) We derive an inference method for this, and propose the NPB-DAA. (3) We show that the NPB-DAA can discover words directly from continuous human speech signals in an unsupervised manner.

Unsupervised Phoneme and Word Discovery From Multiple Speakers Using Double Articulation Analyzer and Neural Network With Parametric Bias

“…Hierarchical Dirichlet process hidden language model (HDP-HLM) is a probabilistic generative model that models double articulation structure (i.e., two-layer hierarchy, a characteristic of human spoken language) (Taniguchi et al, 2016b). Mathematically, HDP-HLM is a natural extension of hierarchical Dirichlet process hidden semi-Markov model (HDP-HSMM), which is a type of generalization of hidden Markov model (Johnson and Willsky, 2013).…”

“…An inference procedure for HDP-HLM was proposed in (Taniguchi et al, 2016b). This procedure is based on the blocked Gibbs sampler for HDP-HSMM proposed by Johnson (Johnson and Willsky, 2013).…”

“…In this paper, we omit the details of the procedure. For further details, please refer to the original paper (Taniguchi et al, 2016b). Sample a word inventory {w i } i=1,2,...,N using sampling importance re-sampling procedure (see Taniguchi et al (2016b)).…”

“…For further details, please refer to the original paper (Taniguchi et al, 2016b). Sample a word inventory {w i } i=1,2,...,N using sampling importance re-sampling procedure (see Taniguchi et al (2016b)). Sample a word model {π W M i }, β W M on the basis of the sampled word inventory {w i } i=1,2,...,N .…”

Extraction of Hierarchical Behavior Patterns Using a Non-parametric Bayesian Approach

“…To segment such data, extraction of temporal patterns in an unsupervised manner is necessary. This has become an active topic in several research fields, such as health care (Zeger et al, 2006), biology (Saeedi et al, 2016), speech recognition (Taniguchi et al, 2016), natural language processing (Heller et al, 2009), and motion analysis (Barbič et al, 2004). Although many methods have been proposed to extract temporal patterns (Keogh et al, 2004), there exists a problem that the number of existing patterns (and consequently, the number of segments) is generally unknown beforehand.…”