An Audio-Visual Whisper Database in Chinese

Abstract

Converting whisper to normal vocalized speech has been a hot research topic in speech signal processing area. A complete and large scale whisper database is a major basis for this task. In this paper, we propose a multimodal whisper database in Chinese mandarin. A total of 103 syllables and 100 sentences were carefully selected. 5 male and 5 female participants pronounced the syllables and sentences in whisper and normal styles respectively, result in 4096 parallel speech utterances and 263, 849 frames of voicing face and lip image sequences. The beginning and ending sample point of each syllable were labeled both for speech signal and voicing face video. The lip region of interest were also extracted and provided in the proposed database. Experiments in various speech conversion tasks in different speech database show the effectiveness of the proposed multimodal whisper speech database.

Identifying languages in a novel dataset: ASMR-whispered speech

“…Whispered speech, also known as unvoiced speech and typically produced with no vocal-cord vibration, is characterized by low-energy (Zhou et al, 2019 ). As opposed to “normal” speech, the speech produced through the use of voiced sounds with harmonic excitation, whispered speech is produced with broad-band noise (Zhou et al, 2019 ), being, for instance, the typical form of communication for individuals diagnosed with aphonia (Zhou et al, 2019 ). In our hypersonic world, whispered speech, which usually requires closeness between speaker and listener (Li, 2011 ), presents an inherent affective component.…”