Three Factors Are Critical in Order to Synthesize Intelligible Noise-Vocoded Japanese Speech

Abstract

Factor analysis (principal component analysis followed by varimax rotation) had shown that 3 common factors appear across 20 critical-band power fluctuations derived from spoken sentences of eight different languages [Ueda et al. (2010). Fechner Day 2010, Padua]. The present study investigated the contributions of such power-fluctuation factors to speech intelligibility. The method of factor analysis was modified to obtain factors suitable for resynthesizing speech sounds as 20-critical-band noise-vocoded speech. The resynthesized speech sounds were used for an intelligibility test. The modification of factor analysis ensured that the resynthesized speech sounds were not accompanied by a steady background noise caused by the data reduction procedure. Spoken sentences of British English, Japanese, and Mandarin Chinese were subjected to this modified analysis. Confirming the earlier analysis, indeed 3–4 factors were common to these languages. The number of power-fluctuation factors needed to make noise-vocoded speech intelligible was then examined. Critical-band power fluctuations of the Japanese spoken sentences were resynthesized from the obtained factors, resulting in noise-vocoded-speech stimuli, and the intelligibility of these speech stimuli was tested by 12 native Japanese speakers. Japanese mora (syllable-like phonological unit) identification performances were measured when the number of factors was 1–9. Statistically significant improvement in intelligibility was observed when the number of factors was increased stepwise up to 6. The 12 listeners identified 92.1% of the morae correctly on average in the 6-factor condition. The intelligibility improved sharply when the number of factors changed from 2 to 3. In this step, the cumulative contribution ratio of factors improved only by 10.6%, from 37.3 to 47.9%, but the average mora identification leaped from 6.9 to 69.2%. The results indicated that, if the number of factors is 3 or more, elementary linguistic information is preserved in such noise-vocoded speech.

Temporal Resolution Needed for Auditory Communication: Measurement With Mosaic Speech

“…We generated a band noise in each critical band, which was amplitude-modulated to make its intensity fluctuation equivalent to that observed in the same frequency band of the original speech signal. This follows basically the procedure to make noise-vocoded speech (Shannon et al, 1995 ; Smith et al, 2002 ; Kishida et al, 2016 ), which is exemplified in Figure 3A . This noise-vocoded speech was almost perfectly intelligible (see Ellermeier et al, 2015 , for related data); it contains linguistic information sufficient for speech perception.…”

“… Examples of noise-vocoded speech (Shannon et al, 1995 ; Smith et al, 2002 ; Ellermeier et al, 2015 ; Kishida et al, 2016 ) and mosaic speech. The same original speech as in Figure 2 was used.…”

Reversal of relationship between impression of voice pitch and height of fundamental frequency: Its appearance and disappearance

“…To measure the independent effects of VTL, f o and its harmonic structure had to be eliminated in the speech samples. Therefore, noise-vocoded speech samples were generated from the original speech samples using white noise as a source signal instead of a modeled glottal sound containing the f o pattern [17,18]. A smoothed spectrum of both long and short VTL voices was then applied to the synthesized experimental speech samples.…”

How sonority appears in speech analyses

“…Thus, the resynthesized speech generally lacks silent parts, making it difficult for us to conduct listening tests utilizing such speech stimuli. Kishida et al (2016) [3], in their study of Japanese speech perception, managed to avoid this problem by modifying the factor analysis so that the acoustically silent point was the origin of the principal component analysis to be performed as the preparatory step for factor analysis. They also smoothed spectra in order to minimize the influence of the harmonic structures observed widely in speech.…”

“…If these power fluctuations are realized by amplitude-modulated band noises in the same frequency bands, a noise-vocoded speech is obtained. This resynthesized speech is almost perfectly intelligible (Ellermeier et al, 2015;Kishida, Nakajima, Ueda & Remijn, 2016) [2,3]. We have been performing a series of analyses and listening experiments from such a viewpoint, but have had no opportunity to give an explanation about how these studies are connected to one another and to previous research.…”