Voice conversion based on Non-negative matrix factorization using phoneme-categorized dictionary

Aihara, Ryo; Nakashika, Toru; Takiguchi, Tetsuya; Ariki, Yasuo

doi:10.1109/icassp.2014.6855137

Cited by 38 publications

(52 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…First, we assume that our NMF approach is advantageous in that it results in a more natural-sounding converted voice compared to conventional statistical VC. The natural sounding converted voice in NMF-based VC has been confirmed [5]. Wu et al [35] applied a spectrum compression factor to NMF-based VC to improve conversion quality.…”

Section: Related Workmentioning

confidence: 93%

Parallel Dictionary Learning for Multimodal Voice Conversion Using Matrix Factorization

Aihara

Masaka

Takiguchi

et al. 2016

Computer and Information Science

Self Cite

View full text Add to dashboard Cite

Parallel dictionary learning for multimodal voice conversion is proposed in this paper. Because of noise robustness of visual features, multimodal feature has been attracted in the field of speech processing, and we have proposed multimodal VC using Non-negative Matrix Factorization (NMF). Experimental results showed that our conventional multimodal VC can effectively converted in a noisy environment, however, the difference of conversion quality between audio input VC and multimodal VC is not so large in a clean environment. We assume this is because our exemplar dictionary is over-complete. Moreover, because of non-negativity constraint for visual features, our conventional multimodal NMF-based VC cannot factorize visual features effectively. In order to enhance the conversion quality of our NMF-based multimodal VC, we propose parallel dictionary learning. Non-negative constraint for visual features is removed so that we can handle visual features which include negative values. Experimental results showed that our proposed method effectively converted multimodal features in a clean environment.

show abstract

Section: Related Workmentioning

confidence: 93%

Parallel Dictionary Learning for Multimodal Voice Conversion Using Matrix Factorization

Aihara

Masaka

Takiguchi

et al. 2016

Computer and Information Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [15], we proposed multimodal NMF-based VC to enhance the noise robustness of our method. The natural-sounding converted voice in NMF-based VC was confirmed in [16]. Wu et al [17] applied a spectrum compression factor to NMF-based VC and improved the conversion quality.…”

Section: Related Workmentioning

confidence: 93%

“…Although the parallel dictionaries are aligned by DTW, there still seems to be a mismatch of alignment. These mismatch degrades the performance of the exemplar-based VC [16]. Second, we assume that the activity matrix contains not only phonetic information but also speaker information.…”

Section: Difference Between Parallel Dictionariesmentioning

confidence: 99%

See 1 more Smart Citation

Small-parallel exemplar-based voice conversion in noisy environments using affine non-negative matrix factorization

Aihara

Fujii

Nakashika

et al. 2015

J AUDIO SPEECH MUSIC PROC.

Self Cite

View full text Add to dashboard Cite

The need to have a large amount of parallel data is a large hurdle for the practical use of voice conversion (VC). This paper presents a novel framework of exemplar-based VC that only requires a small number of parallel exemplars. In our previous work, a VC technique using non-negative matrix factorization (NMF) for noisy environments was proposed. This method requires parallel exemplars (which consist of the source exemplars and target exemplars that have the same texts uttered by the source and target speakers) for dictionary construction. In the framework of conventional Gaussian mixture model (GMM)-based VC, some approaches that do not need parallel exemplars have been proposed. However, in the framework of exemplar-based VC for noisy environments, such a method has never been proposed. In this paper, an adaptation matrix in an NMF framework is introduced to adapt the source dictionary to the target dictionary. This adaptation matrix is estimated using only a small parallel speech corpus. We refer to this method as affine NMF, and the effectiveness of this method has been confirmed by comparing its effectiveness with that of a conventional NMF-based method and a GMM-based method in noisy environments.

show abstract

“…Ce paradigme de conversion de la voix présente cependant des limites importantes et bien connues : les effets de sur-apprentissage et de moyennage relatifsà la modélisation statistique (Toda et al, 2007) qui conduità une dégradation de la voix convertie, et la nécessité de construire des bases de données parallèles extrêmement restrictives et non souhaitées pour des applications réelles. Pour répondreà ces limitations, des algorithmes de conversionà partir d'unités -ou d'exemples réels -ontété récemment proposés (Sündermann et al, 2006;Wu et al, 2013;Aihara et al, 2014;Jin et al, 2016). Tout d'abord, la conversion vocaleà partir de sélection et de concaténation d'unités spectrales présente l'avantage de préserver les caractéristiques et la dynamique d'origine de la voix cible dans la mesure où elle repose uniquement sur l'utilisation d'unités vocales réelles.…”

Section: Introductionunclassified

Conversion d'Identité de la Voix Chantée par Sélection et Concaténation d'Unités Spectrales

Obin¹,

Pham²,

Röebel³

2018

XXXIIe Journées D’Études Sur La Parole

View full text Add to dashboard Cite

RÉSUMÉCet article présente un algorithme de sélection d'unités spectrales pour la conversion de l'identité de la voix chantéeà partir de bases de données non parallèles. Les algorithmes de conversion basés sur des unités de parole présentent des avantages importants pour la conversion de l'identité vocale : la conversion vocale par sélection d'unités permet la préservation des caractéristiques originales de la voix cible, en utilisant des unités réelles ; et la segmentation en unités linguistiques permet d'apprendre la conversionà partir d'enregistrements de la voix cible non nécessairement alignés avec ceux de la voix source. La contribution principale de cet article est de réaliser la sélection des unités spectrales de la voix cible en fonction de plusieurs facteurs : acoustique, linguistique (phonèmes) et musicaux (hauteur, intensité et durée). Pour ce faire, la sélection de la séquence d'unités d'enveloppe spectrale estétablie comme un problème d'optimisationà partir d'une fonction de coût multiple qui comprend la distorsion spectrale des chanteurs source et cible ainsi que les différences de hauteur, d'intensité et de durée des unités spectrales correspondantes. L'objectif est de guider la sélection vers des enveloppes spectrales du chanteur cible partageant un contexte musical similaire avec celles du chanteur source. Il est montré lors d'une expérience perceptive que l'algorithme proposé améliore le naturel de la conversion et la similarité avec la voix cible. ABSTRACTThis paper presents a unit-selection algorithm for non-parallel singing voice conversion. Unit-based algorithms presents important advantages for voice conversion : the speech segmentation into linguistic units allows the possibility to learn the conversion from on-the-fly databases of the target voice not necessarily aligned to the source voice, and unit-selection voice conversion allows the preservation of the original characteristics of the target voice, by using real units. The main idea of this paper is that the spectral envelopes of a speaker vary according to multiple factors : linguistics (phonemes), and musical (pitch, intensity, and duration). Accordingly, the selection of the sequence of spectral envelope units is established as a multi-target optimization problem, including the spectral distortion of the source and target singers, and the pitch, intensity, and duration differences of the corresponding spectral envelopes. The objective is to guide the selection towards spectral envelopes of the source and target singers sharing a similar musical context. It is shown that the proposed algorithm improves conversion naturalness and target similarity.MOTS-CLÉS : conversion de l'identité vocale, voix chantée, conversion non-parallèle, sélection d'unités, optimisation multi-cible.

show abstract

Voice conversion based on Non-negative matrix factorization using phoneme-categorized dictionary

Cited by 38 publications

References 16 publications

Parallel Dictionary Learning for Multimodal Voice Conversion Using Matrix Factorization

Parallel Dictionary Learning for Multimodal Voice Conversion Using Matrix Factorization

Small-parallel exemplar-based voice conversion in noisy environments using affine non-negative matrix factorization

Conversion d'Identité de la Voix Chantée par Sélection et Concaténation d'Unités Spectrales

Contact Info

Product

Resources

About