Can We Use Speaker Embeddings On Spontaneous Speech Obtained From Medical Conversations To Predict Intelligibility?

Abstract

The automatic prediction of speech intelligibility is a recurrent problem in the context of pathological speech. Despite recent developments, these systems are normally applied to specific speech tasks recorded in clean conditions that do not necessarily reflect a healthcare environment. In the present paper, we intend to test the reliability of an intelligibility predictor on data obtained in clinical conditions, in the specific case of head and neck cancer. In order to do so, we present a system based on speaker embeddings trained on a multi-task methodology to simultaneous predict speech intelligibility and speech disorder severity. The results obtained on the different evaluation tasks display correlations as high as 0.891 on a hospital patient set, showing robustness to the type of speech material used in these automatic assessments. Moreover, the usage of spontaneous speech during the evaluation shed light on an understudied, but with more ecological validity, type of speech material which displayed promising results. The reliability displayed across the different tasks suggests a direct deployment of the developed systems in a hospital setting.

SAMI: an M-Health application to telemonitor intelligibility and speech disorder severity in head and neck cancers

“…In the present work, we made use of the x-vector speaker embeddings, widely used in pathological speech assessment (Jeancolas et al, 2021;Kotarba and Kotarba, 2021;Scheuerer et al, 2021). Concerning the automatic prediction of speech intelligibility, these embeddings have outperformed other embedding types such as i-vectors by more than 10% in correlation on a passage reading task (Quintas et al, 2020), and also the more recent ECAPA_TDNN by as much as 13% on a spontaneous and reading speech tasks (Quintas et al, 2023b), making them the state-of-the-art for this particular task. Similarly to previous works, we intend to use them as features for a subsequent signal processing chain.…”

“…These measures are not only clinically relevant, but can also be combined linearly to predict speech intelligibility (Bodt et al, 2002), an aspect that could add an extra layer of interpretability to our system. Furthermore, SAMI could also be adapted to predict the severity of speech disorder and intelligibility measures from a real-life interactions between doctor-patient or even caregiver-patient, an aspect currently underexplored that mainly addresses the assessment of spontaneous speech in clinical contexts (Quintas et al, 2023b).…”