Whisper to Normal Speech Based on Deep Neural Networks with MCC and F0 Features

Lian, Hailun; Hu, Yuting; Zhou, Jian; Wang, Huabin; Tao, Liang

doi:10.1109/icdsp.2018.8631888

Cited by 6 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some version of a dataset derived from the TIMIT corpus was used in all cases included here. Specifically, whispered TIMIT (wTimit) 1 was used by Niranjan et al (2020), Patel et al (2021), Parmar et al (2019), andPatel et al (2019), while CSTR-NAM-TIMIT Plus was used by Gao et al (2021), Lian et al (2019a), Malaviya et al (2020), Pang et al (2020), Yu et al (2019), and Lian et al (2019b). The wTIMIT dataset uses the prompts in TIMIT, a well-known corpus often used for benchmarking in speech recognition, including 450 phonetically balanced sentences both in normal and whispered speech.…”

Section: Datasetsmentioning

confidence: 99%

“…A higher MCD value indicates a greater difference between the converted and the reference speech (lower is better). Additionally, Short-Time Objective Intelligibility (STOI) was also used in Lian et al (2019a), Pang et al (2020), Yu et al (2019), andLian et al (2019b) and Perceptual Evaluation of Speech Quality was also used in Lian et al (2019a), Pang et al (2020), andYu et al (2019). Finally, the two remaining papers used entirely different metrics from the rest.…”

Section: Objective Metricsmentioning

confidence: 99%

“…Additionally, in Patel et al (2021), the Kullback-Leibler Divergence (KLD) and Jensen-Shannon Divergence (JSD) between predicted and original F0 for speaker-specific tasks were also considered. In Niranjan et al (2020), F0 is not estimated and in Pang et al (2020), Yu et al (2019), andLian et al (2019b), while the F0 estimation is discussed and accompanied with illustrations of the F0 curve, no objective metric is presented.…”

Section: Objective Metricsmentioning

confidence: 99%

See 2 more Smart Citations

Machine Learning Approaches for Whisper to Normal Speech Conversion

Oliveira

2022

UPjeng

View full text Add to dashboard Cite

Whispered speech is a mode of speech that differs from normal speech due to the absence of a periodic component, namely the Fundamental Frequency that characterizes the pitch, among other spectral and temporal differences. Much attention has been given in recent years to the application of Machine Learning techniques for voice conversion tasks. The whisper-to-normal speech conversion is particularly challenging, however, especially with respect to the Fundamental Frequency estimation. Based on the most recent literature, this survey assesses the state-of-the-art regarding Machine Learning based whisper-to-normal speech conversion, identifying trends both on modeling and training approaches. The proposed solutions include Generative Adversarial Network based, Autoencoder based and Bidirectional Long Short-Term Memory based frameworks, among other Deep Neural Network based architectures. In addition to Parallel versus Non-Parallel training, time-alignment requirements and strategies, datasets, vocoder usage, as well as both objective and subjective evaluation metrics are also covered by the present survey.

show abstract

Section: Datasetsmentioning

confidence: 99%

Section: Objective Metricsmentioning

confidence: 99%

Section: Objective Metricsmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning Approaches for Whisper to Normal Speech Conversion

Oliveira

2022

UPjeng

View full text Add to dashboard Cite

show abstract

“…Voice conversion (VC) is a technique for converting one voice into another voice with a different timbre under the condition of keeping linguistic information. In recent years, there have been many applications based on voice conversion, such as vocal conversion [1][2][3][4][5][6][7][8][9][10][11][12][13], singing voice conversion [14][15][16], emotion conversion [17,18], speech style conversion [19,20], conversion of whispers to normal voices [21,22], conversion of singing skills [23], voice correction [24], and so on. However, there are few studies on the conversion of human voices to musical instruments.…”

Section: Introductionmentioning

confidence: 99%

CycleGAN-Based Singing/Humming to Instrument Conversion Technique

Lai

Wang²,

2022

Electronics

View full text Add to dashboard Cite

In this research, singing/humming to instrument conversion techniques are proposed. In humming to instrument, two models based on cycle-consistent adversarial networks (CycleGAN) on viola are experimented. From the objective and subjective evaluations conducted, the converted audio is more similar to viola compared to humming, and the quality of the converted sound is fair to listeners. In singing to instrument, to fix the problem of the gap between singing and instrument, a dual conversion model consisting of singing to humming and humming to instrument is proposed. The objective and subjective experimental results show that the dual conversion has better converted audio quality than conversion by singing to instrument directly.

show abstract