Goal-Directed Exploration for Learning Vowels and Syllables: A Computational Model of Speech Acquisition

Abstract

Infants learn to speak rapidly during their first years of life, gradually improving from simple vowel-like sounds to larger consonant-vowel complexes. Learning to control their vocal tract in order to produce meaningful speech sounds is a complex process which requires to learn the relationship between motor and sensory processes. In this paper, a computational framework is proposed that models the problem of learning articulatory control for a physiologically plausible 3-D vocal tract model using a developmentally-inspired approach. The system babbles and explores efficiently in a low-dimensional space of goals that are relevant to the learner in its synthetic environment. The learning process is goal-directed and self-organized, and yields an inverse model of the mapping between sensory space and motor commands. This study provides a unified framework that can be used for learning static as well as dynamic motor representations. The successful learning of vowel and syllable sounds as well as the benefit of active and adaptive learning strategies are demonstrated. Categorical perception is found in the acquired models, suggesting that the framework has the potential to replicate phenomena of human speech acquisition.

Reverse Engineering Language Acquisition with Child-Centered Long-Form Recordings

“…To take a specific example, Warlaumont et al (2011)'s model has an articulatory component (that generates the child's output in terms of gestures) as well as a perceptual auditory component (that captures patterns in the input as well as the auditory consequences of the child's production) -but note that the articulatory component does not include biophysical constraints per se. Approaches with realistic models of the developing vocal tract are rare (but see Philippsen 2021).…”

Application of Big Data Mining Technology in the Digital Construction of Vocal Music Teaching Resource Library

“…This subject first needs to understand the content related to piano teaching software in various fields and then integrate the knowledge in various fields to carry out the research of this subject [10]. The ultimate purpose of this subject is to realize a software, so the computer aspect involves the most content, in the aspect of music theory.…”

Exploration strategies for articulatory synthesis of complex syllable onsets

“…Babbling during early vocal learning has often been simulated as a goal-directed or imitative process, usually involving a set of auditory objectives [7][8][9][10][11]. This type of exploration is also considered central to finding appropriate inverse models during sensorimotor learning in general [12,13].…”

“…The correspondence problem is one of associating articulatory gestures obtained for an acoustic reference to linguistic contexts [11,16,17]. We have argued elsewhere, based on the well-known finding that language-oriented speech perception precedes the onset of canonical babbling in infants [18], that these problems can be addressed by a language-oriented auditory perceptual mapping derived from linguistically grounded multi-speaker speech stimuli [19].…”