Combined X-ray and facial videos for phoneme-level articulator dynamics

Chen, Hui; Wang, Lan; Liu, Wenxi; Heng, Pheng‐Ann

doi:10.1007/s00371-010-0434-1

Cited by 10 publications

(1 citation statement)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both English talking head and Mandarin talking head have been proposed in our previous studies [7], [22], [23], [45]- [47]. For Mandarin 3-D external and internal articulator dynamics, articulatory data specifying in Mandarin production were collected with Carstens AG-501 EMA [22], [23].…”

Section: A Mandarin 3-d Talking Head With Internal Articulator Dynamicsmentioning

confidence: 99%

Investigations on Mandarin Aspiratory Animations Using an Airflow Model

Chen

Wang

Chen

et al. 2017

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

Various 3-D talking heads have been developed lately for language learning, with both external and internal articulatory movements being visualized to guide learning. Mandarin pronunciation animation is challenging due to its confusable stops and affricates with similar places of articulation. Until now, less attention has been paid to the bio-signal information of aspiratory airflow, which is essential in distinguishing Mandarin consonants. This study fills a research gap by presenting the quantitative analyses of airflow, and then designing an airflow model for a 3-D pronunciation system. The airflow information was collected by Phonatory Aerodynamic System (PAS), so that confusable consonants in Mandarin could be discerned by means of airflow rate, peak airflow rate, airflow duration, and peak time. Based on the airflow parameters, an airflow model using the physical equation of fluid flow was proposed and solved, which was then combined and synchronized with the existing 3-D articulatory model. Therefore, the new multimodal system was implemented to synchronously exhibit the airflow motions and articulatory movements of uttering Mandarin syllables. Both an audio-visual perception test and a pronunciation training study were conducted to assess the effectiveness of our system. Perceptual results indicated that identification accuracy was improved for both native and non-native groups with the help of airflow motions, while native perceivers exhibited higher accuracy due to long-term language experience. Moreover, our system helped Japanese learners of Mandarin enhance their production skills of Mandarin aspirated consonants, reflected by higher gain values of voice onset time (VOT) after training.

show abstract