Segmental intelligibility of synthetic speech produced by rule

Abstract

This paper reports the results of an investigation that employed the modified rhyme test (MRT) to measure the segmental intelligibility of synthetic speech generated automatically by rule. Synthetic speech produced by ten text-to-speech systems was studied and compared to natural speech. A variation of the standard MRT was also used to study the effects of response set size on perceptual confusions. Results indicated that the segmental intelligibility scores formed a continuum. Several systems displayed very high levels of performance that were close to or equal to scores obtained with natural speech; other systems displayed substantially worse performance compared to natural speech. The overall performance of the best system, DECtalk--Paul, was equivalent to the data obtained with natural speech for consonants in syllable-initial position. The findings from this study are discussed in terms of the use of a set of standardized procedures for measuring intelligibility of synthetic speech under controlled laboratory conditions. Recent work investigating the perception of synthetic speech under more severe conditions in which greater demands are made on the listener's processing resources is also considered. The wide range of intelligibility scores obtained in the present study demonstrates important differences in perception and suggests that not all synthetic speech is perceptually equivalent to the listener.

Effects of intelligibility on working memory demand for speech perception

“…For the purpose of comparison with older TTS systems, 20 listeners were tested using the modified rhyme test. Results showed a significant difference in percent correct responses [t(19) 11.13, p .001], with 93.9% correct identification for the Cepstral David voice and 70.3% for rsynth, comparable to those shown for DECtalk (96.75%) and Votrax (72.56%), respectively, as reported by Logan, Greene, and Pisoni (1989).…”

Perception of Synthetic Speech

“…By comparing the results obtained in the closed-and open-response versions of the MRT, we. were able to .obtain a great deal of useful informatiQn abQut the sources of errQr a particular system [LGP89). Indeed, detailed analyses of the stimulus-response cQnfusiQns provided knQwledge about the specific rules used to generate segmental contrasts in particular phQnetic environments and how to modify them to imJ)I'OYce,', intelligibility [Kla87).…”

“…In recent years, new tests specifically for the assessment of synthetic speech have been developed using this approach [BP89,CG89,SAMW89]. Some examples of data obtained using in the MRT from [LGP89] for 10 text-to-speech systems are shown in figure 43.1. The intelligibility data shown here reveal a wide range of performance levels across different synthesis systems.…”

Flexible Speech Translation Systems

“…Although objective measures can be calculated for speech synthesizers for text analysis, lexicon and letter-to-sound rule coverage and accuracy, ultimately native listeners must be involved. Three levels of sentence type may be generated and listened to by 5-10 native speakers: 1) phonetic confusable words, following the Diagnostic Rhyme Test [37]; 2) semantically unpredictable sentences (SUS) [38], where templates are used to generate sentences from simple part of speech classes; 3) in domain sentences, which are checked for intelligibility. These tests are designed to be diagnostic, in other words, to identify lexical and labeling problems which can then be fixed.…”