Characterization of inter-speaker articulatory variability: A two-level multi-speaker modelling approach based on MRI data

Abstract

Speech communication relies on articulatory and acoustic codes shared between speakers and listeners despite inter-individual differences in morphology and idiosyncratic articulatory strategies. This study addresses the long-standing problem of characterizing and modelling speaker-independent articulatory strategies and inter-speaker articulatory variability. It explores a multi-speaker modelling approach based on two levels: statistically-based linear articulatory models, which capture the speakerspecific articulatory variability on the one hand, are in turn controlled by a speaker model, which captures the inter-speaker variability on the other hand. A low dimensionality speaker model is obtained by taking advantage of the inter-speaker correlations between morphology and strategy. To validate this approach, contours of the vocal tract articulators were manually segmented on midsagittal MRI data recorded from 11 French speakers uttering 62 vowels and consonants. Using these contours, multi-speaker models with 14 articulatory components and two morphology and strategy components led to overall variance explanations of 66%-69% and root-mean-square errors of 0.36-0.38 cm obtained in leave-one-out procedure over the speakers. Results suggest that inter-speaker variability is more related to the morphology than to the idiosyncratic strategies and illustrate the adaptation of the articulatory components to the morphology. V

Automatic vocal tract landmark localization from midsagittal MRI data

“…This region is characterized by a high inter-and intra-speaker variability, both in the space and time domains. Analyzing and modeling the shape of the vocal tract articulators is therefore crucial for speech production research [1][2][3] and for diagnosis and therapy of related disorders, including speech disorders 4,5 , velopharyngeal insufficiency 6 and swallowing dysfunctions 7 . The vocal tract area extends from the glottis to the lips and is surrounded by various structures such as the larynx, the epiglottis, the velum, the tongue, and the upper and lower lips.…”

“…Articulatory speech production studies often rely on midsagittal images of the vocal tract area and Magnetic Resonance Imaging (MRI) constitutes in this approach an essential modality [8][9][10] . Identifying landmarks of the vocal tract area on these images has always been done manually or as a byproduct of manual segmentation 3,11 . If such a manual approach fulfilled the needs until nowadays, the exploding number of data in this field due to the rise of real-time MRI 12 and the recent progress in data science call undoubtedly for an automatic approach.…”

“…One can finally also mention the use of landmark identification for research on non-human Methods Data. The study considers static midsagittal MRI recorded between 2002 and 2011 from 9 French speakers (5 males, 4 females), referred to as subjects in this study, sustaining 62 different articulatory positions, also referred to as classes in the context of machine learning, designed to be representative of the French phonemic repertoire 3,36 . This study does not include any human experiments more than the use of non-invasive MRI data collection mentioned above.…”

“…The images have been recorded either on a 1.5 or on a 3 Tesla MRI scanner and have a field of view of 256 × 256 mm 2 and a resolution of 1 mm per pixel. Note that two speakers have been discarded in comparison to Serrurier et al 3 and Valdés Vargas 36 due to the significantly lower quality of the images, the different fields of view and the different sizes of the images. 21 anatomical landmarks relevant to the study of the speech articulations have been identified.…”

“…They have been determined for each subject by contrast with soft tissues on an articulation acquired on purpose and reported on the other images using their relative position with the hard palate. Please refer to Serrurier et al 3 for further information regarding this procedure as well as the data collection and processing.…”