Speech-to-Singing Voice Conversion: The Challenges and Strategies for Improving Vocal Conversion Processes

Vijayan, Karthika; Li, Haizhou; Toda, Tomoki

doi:10.1109/msp.2018.2875195

Cited by 20 publications

(18 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Speech-to-singing conversion is the task of transforming the spoken lyrics of a song into singing, while retaining the identity of the speaker and the linguistic content [33]. In [20], the authors proposed a method to transform speech into singing, by modifying the pitch contour, the duration of the phonemes and the spectrum according to the analysis of the features of the singing voice.…”

Section: Speech-to-singingmentioning

confidence: 99%

A unit selection text-to-speech-and-singing synthesis framework from neutral speech: proof of concept

Freixes

Álías

Socoró

2019

J AUDIO SPEECH MUSIC PROC.

View full text Add to dashboard Cite

Text-to-speech (TTS) synthesis systems have been widely used in general-purpose applications based on the generation of speech. Nonetheless, there are some domains, such as storytelling or voice output aid devices, which may also require singing. To enable a corpus-based TTS system to sing, a supplementary singing database should be recorded. This solution, however, might be too costly for eventual singing needs, or even unfeasible if the original speaker is unavailable or unable to sing properly. This work introduces a unit selection-based text-to-speech-and-singing (US-TTS&S) synthesis framework, which integrates speech-to-singing (STS) conversion to enable the generation of both speech and singing from an input text and a score, respectively, using the same neutral speech corpus. The viability of the proposal is evaluated considering three vocal ranges and two tempos on a proof-of-concept implementation using a 2.6-h Spanish neutral speech corpus. The experiments show that challenging STS transformation factors are required to sing beyond the corpus vocal range and/or with notes longer than 150 ms. While score-driven US configurations allow the reduction of pitch-scale factors, timescale factors are not reduced due to the short length of the spoken vowels. Moreover, in the MUSHRA test, text-driven and score-driven US configurations obtain similar naturalness rates of around 40 for all the analysed scenarios. Although these naturalness scores are far from those of vocaloid, the singing scores of around 60 which were obtained validate that the framework could reasonably address eventual singing needs.

show abstract

Section: Speech-to-singingmentioning

confidence: 99%

A unit selection text-to-speech-and-singing synthesis framework from neutral speech: proof of concept

Freixes

Álías

Socoró

2019

J AUDIO SPEECH MUSIC PROC.

View full text Add to dashboard Cite

show abstract

“…A singing voice utilizes vocal cord muscle tension to regulate the pitch and duration. Its average intensity is thus beyond that of speech, its dynamic vary is more significant, and its tone is usually totally different from that of speech [ 20 ].…”

Section: Introductionmentioning

confidence: 99%

Singing Voice Detection: A Survey

Monir

Kostrzewa

Mrozek

2022

Entropy

View full text Add to dashboard Cite

Singing voice detection or vocal detection is a classification task that determines whether there is a singing voice in a given audio segment. This process is a crucial preprocessing step that can be used to improve the performance of other tasks such as automatic lyrics alignment, singing melody transcription, singing voice separation, vocal melody extraction, and many more. This paper presents a survey on the techniques of singing voice detection with a deep focus on state-of-the-art algorithms such as convolutional LSTM and GRU-RNN. It illustrates a comparison between existing methods for singing voice detection, mainly based on the Jamendo and RWC datasets. Long-term recurrent convolutional networks have reached impressive results on public datasets. The main goal of the present paper is to investigate both classical and state-of-the-art approaches to singing voice detection.

show abstract

“…As discussed in [6], even though speech and singing signals are produced by same vocal production system and consequently share many properties, their production takes place within very different settings. The key challenges for achieving conversion involve aligning two very different signals (equivalent to modelling phoneme durations), imposing the required melody on speech without losing linguistic content and speaker identity.…”

Section: Introductionmentioning

confidence: 99%

“…The key challenges for achieving conversion involve aligning two very different signals (equivalent to modelling phoneme durations), imposing the required melody on speech without losing linguistic content and speaker identity. Previous works on STS conversion can be broadly categorized into two approaches [6]:…”

Section: Introductionmentioning

confidence: 99%

Speech-To-Singing Conversion in an Encoder-Decoder Framework

Parekh

Rao

Yang

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper our goal is to convert a set of spoken lines into sung ones. Unlike previous signal processing based methods, we take a learning based approach to the problem. This allows us to automatically model various aspects of this transformation, thus overcoming dependence on specific inputs such as high quality singing templates or phoneme-score synchronization information. Specifically, we propose an encoder-decoder framework for our task. Given timefrequency representations of speech and a target melody contour, we learn encodings that enable us to synthesize singing that preserves the linguistic content and timbre of the speaker while adhering to the target melody. We also propose a multi-task learning based objective to improve lyric intelligibility. We present a quantitative and qualitative analysis of our framework.

show abstract

Speech-to-Singing Voice Conversion: The Challenges and Strategies for Improving Vocal Conversion Processes

Cited by 20 publications

References 24 publications

A unit selection text-to-speech-and-singing synthesis framework from neutral speech: proof of concept

A unit selection text-to-speech-and-singing synthesis framework from neutral speech: proof of concept

Singing Voice Detection: A Survey

Speech-To-Singing Conversion in an Encoder-Decoder Framework

Contact Info

Product

Resources

About