Comprehension of natural and synthetic speech: effects of predictability on the verification of sentences controlled for intelligibility

Abstract

A sentence verification task (SVT) was used to study the effects of sentence predictability on comprehension of natural speech and synthetic speech that was controlled for intelligibility. Sentences generated using synthetic speech were matched on intelligibility with natural speech using results obtained from a separate sentence transcription task. In the main experiment, the sentence verification task included both true and false sentences that varied in predictability. Results showed differences in verification speed between natural and synthetic sentences, despite the fact that these materials were equated for intelligibility. This finding suggests that the differences in perception and comprehension between natural and synthetic speech go beyond segmental intelligibility as measured by transcription accuracy. The observed differences in response times appear to be related to the cognitive processes involved in understanding and verifying the truth value of short sentences. Reliable effects of predictability on error rates and response latencies were also observed. High-predictability sentences displayed lower error rates and faster response times than low-predictability sentences. However, predictability did not have differential effects on the processing of synthetic speech as expected. The results demonstrate the need to develop new measures of sentence comprehension that can be used to study speech communication at processing levels above and beyond those indexed through transcription tasks, or forced-choice intelligibility tests such as the Modified Rhyme Test (MRT) or the Diagnostic Rhyme Test (DRT).Over the past six years, numerous studies on the perception of synthetic speech have been conducted in our laboratory at Indiana University (Pisoni, 1982;Pisoni, Nusbaum & Greene, 1985). The bulk of these studies have focused on measures of segmental intelligibility, such as identification of isolated words and recognition of words in sentences (e.g., Egan, 1948;House, Williams, Hecker & Kryter, 1965;Nye & Gaitenby, 1973). Results from these studies of phoneme and word perception have shown that synthetic speech is consistently less intelligible than natural speech (Greene, Logan & Pisoni, 1986). This finding was observed for a variety of synthesis systems ranging from very low-quality, low-intelligibility products, such as the ECHO and Votrax Type N Talk, to extremely natural sounding speech with very high intelligibility such as DECtalk and the Prose 2000.Most perceptual studies dealing with segmental intelligibility have not addressed the issue of comprehension processes involved in understanding the linguistic content of the message. In tests of segmental intelligibility, such as the ones we have carried out, subjects are not NIH Public Access Author ManuscriptComput Speech Lang. Author manuscript; available in PMC 2012 December 05.Published in final edited form as:Comput Speech Lang. 1987 ; 2(3-4): 303-320. NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author Manuscript required to e...

The effects of familiarization on intelligibility and lexical segmentation in hypokinetic and ataxic dysarthria

“…But these perceptual capabilities have limits, and cognitive effort increases and accuracy decreases as the acoustic information becomes degraded or unreliable ͑Munro and Derwing, 1995;Pisoni et al, 1987͒. This is precisely the case with the perception of dysarthric speech.…”

Perception of Synthetic Speech

“…shown in figure 43.3 indicated that subjects were consistently faster in responding, to natural speech than to synthetic speech. For both natural and synthetic speec~;: responses were faster for high-predictability sentences than for low-predictabjIi,tY:':; sentences [PMD87]. The results showed that although the sentences were intelligible, even high-quality synthetic speech is not perceived in the same as natural speech.…”

The Effects of Familiarization on the Comprehension of Synthetic Speech in Telephone Communication

“…Although some have found rule-based synthetic speech to attain performance levels approximating those of natural and digitized speech, results from other studies have indicated inferior comprehension performance (Moody and Joost, 1986;Pisoni and Hunnicutt, 1980;Pisoni, Manous, and Dedina, 1987;Schwab, Nusbaum and Pisoni, 1985). This difference in comprehension performance between synthetic and natural speech may be mediated by the amount of familiarization, which several researchers have found to enhance the intelligibility and comprehension performance of synthetic speech, at times approaching levels comparable to natural speech (Herlong and Williges, 1988;Pisoni and Hunnicutt, 1980;Stecyk-Ramos, Maddox and Strybel, 1988).…”

“…A total of 64 sentences were selected from a study performed by Pisoni, Manous, and Dedina (1987). In that study, a group of 40 introductory psychology students was asked to create true and false sentences by providing the final word to complete a series of sentence frames.…”