Partial compensation for coarticulatory vowel nasalization across concatenative and neural text-to-speech

Abstract

This study investigates the perception of coarticulatory vowel nasality generated using different text-to-speech (TTS) methods in American English. Experiment 1 compared concatenative and neural TTS using a 4IAX task, where listeners discriminated between a word pair containing either both oral or nasalized vowels and a word pair containing one oral and one nasalized vowel. Vowels occurred either in identical or alternating consonant contexts across pairs to reveal perceptual sensitivity and compensatory behavior, respectively. For identical contexts, listeners were better at discriminating between oral and nasalized vowels in neural than in concatenative TTS for nasalized same-vowel trials, but better discrimination for concatenative TTS was observed for oral same-vowel trials. Meanwhile, listeners displayed less compensation for coarticulation in neural than in concatenative TTS. To determine whether apparent roboticity of the TTS voice shapes vowel discrimination and compensation patterns, a “roboticized” version of neural TTS was generated (monotonized f0 and addition of an echo), holding phonetic nasality constant; a ratings study (experiment 2) confirmed that the manipulation resulted in different apparent roboticity. Experiment 3 compared the discrimination of unmodified neural TTS and roboticized neural TTS: listeners displayed lower accuracy in identical contexts for roboticized relative to unmodified neural TTS, yet the performances in alternating contexts were similar.

The clear speech intelligibility benefit for text-to-speech voices: Effects of speaking style and visual guise

“…Given that devices are perceived as less communicatively competent than humans (Cowan et al, 2015;Cohn et al, 2022), looking at a device may trigger this stereotype and lower comprehension. More broadly, this finding builds on work showing that socio-indexical information and speech perception are intertwined (e.g., D 'Onofrio, 2015) and contributes to research indicating that people have distinct mental representations for humans and devices, which affect speech perception (e.g., Zellou et al, 2021).…”

Listener beliefs and perceptual learning: Differences between device and human guises

“…Second, an echo was added (delay: 0.01 s; 0.5 Pa). Listeners associate flattened pitch and echo with 'robot' voices (Wilson & Moore 2017), and prior work has shown that this procedure for resynthesis yields speech that is rated as significantly more robotic-sounding than unmodified neural TTS (Zellou, Cohn, & Block 2021).…”

“…The latter manipulation follows an approach taken in work exploring the role of 'voice anthropomorphism' in speech perception (e.g. Cowan et al 2015, Zellou, Cohn, & Block 2021.…”

Prosodic focus marking for voice-AI and human addressees

“…Voice-AI assistants are an apt addressee for investigating the rational listener hypothesis as they are rated as 'less communicatively competent' than adult human interlocutors (Cohn et al, 2022), display many errors in recognition (e.g., 20-30% word error rate in Koenecke et al, 2020) and demonstrate difficulties extracting meaning ('natural language understanding') (Beneteau et al, 2019). Additionally, the text-to-speech (TTS) output they produce is often perceived as 'choppy' (Doyle et al, 2019;Zellou, Cohn, & Block, 2021). Indeed, recent work has shown that people modify their speech in distinct ways for voice-AI addressees (for a review, see Cohn et al, 2022), with the most prominent acoustic differences being in the prosodic domain (Cohn et al, 2022;Raveh et al, 2019;Siegert et al, 2019).…”