Articulatory Controllable Speech Modification Based on Statistical Inversion and Production Mappings

Tobing, Patrick Lumban; Kobayashi, Kenzo; Toda, Tomoki

doi:10.1109/taslp.2017.2753583

Cited by 10 publications

(13 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…VC research was originally started to develop a speaker individuality conversion technique enabling a source speaker to change his/her speaker individuality to that of another target speaker while preserving the linguistic content [2]. The conversion frameworks of VC have been adopted into other research objectives such as speech recovery for people with speech disorders [3], singing style conversion [4], nonnative to native speaker conversion [5], and speech to articula-tory mapping [6] to make it possible to implement augmented speech communications. Towards the practical use of these VC applications, it is essential to improve fundamental VC techniques.…”

Section: Introductionmentioning

confidence: 99%

sprocket: Open-Source Voice Conversion Software

Kobayashi¹,

Toda²

2018

EasyChair Preprints

Self Cite

View full text Add to dashboard Cite

Statistical voice conversion (VC) is a technique to convert specific non-or paralinguistic information while keeping linguistic information unchanged, and speaker conversion has been studied as a typical application of VC for a few decades. To better understand various VC techniques using a freely available common dataset, the Voice Conversion Challenge (VCC) was launched in 2016 and the 2nd challenge was held in 2018. As one of the baseline systems for VCC 2018, we developed open-source VC software called "sprocket", in which not only conventional techniques, such as a trajectory-based conversion method using a Gaussian mixture model (GMM) and a vocoderbased conversion framework but also recently developed techniques, such as a vocoder-free VC framework, have been implemented. Using sprocket, it is possible to 1) easily reproduce converted voices using the VCC datasets and 2) develop VC systems using other parallel speech datasets with fundamental VC functions, such as acoustic feature extraction, time alignment between the source and target features, GMM training, feature conversion, and waveform generation. In this paper, we describe 1) the technical details and use of sprocket, 2) the development of the baseline systems for the HUB and SPOKE tasks of VCC 2018 using sprocket, and 3) the performance of sprocket as a VC system by demonstrating results for our developed baseline systems in VCC 2018.

show abstract

Section: Introductionmentioning

confidence: 99%

sprocket: Open-Source Voice Conversion Software

Kobayashi¹,

Toda²

2018

EasyChair Preprints

Self Cite

View full text Add to dashboard Cite

show abstract

“…The capability of handling the speaker characteristics within a speech signal has great potential to be employed in real-world applications. Indeed, this so-called voice conversion (VC) framework has been used in several works, such as, singing voice conversion [1,2], body-conducted speech conversion [3], speech signal recovery [4,5], and speech modification [6]. The growing interest in VC development motivated many researchers around the world to conceive the 1 st Voice Conversion Challenge in 2016 [7].…”

Section: Introductionmentioning

confidence: 99%

NU Voice Conversion System for the Voice Conversion Challenge 2018

Tobing¹,

Wu²,

Hayashi³

et al. 2018

The Speaker and Language Recognition Workshop (Odyssey 2018)

Self Cite

View full text Add to dashboard Cite

This paper presents the NU (Nagoya University) voice conversion (VC) system for the HUB task of the Voice Conversion Challenge 2018 (VCC 2018). The design of the NU VC system can basically be separated into two modules consisting of a speech parameter conversion module and a waveformprocessing module. In the speech parameter conversion module, a deep learning framework is deployed to estimate the spectral parameters of a target speaker given those of a source speaker. Specifically, a deep neural network (DNN) and a deep mixture density network (DMDN) are used as the deep model structure. In the waveform-processing module, given the estimated spectral parameters and linearly transformed F0 parameters, the converted waveform is generated using a WaveNet-based vocoder system. To use the WaveNet-based vocoder, there are several generation flows based on an analysissynthesis framework to obtain the speech parameter set, on the basis of which a system selection process is performed to select the best one in an utterance-wise manner. The results of VCC 2018 ranked the NU VC system in second place with an overall mean opinion score (MOS) of 3.44 for speech quality and 85% accuracy for speaker similarity.

show abstract

“…Related research can be divided into two categories. The first centres on a statistical mapping from articulatory features to acoustic features [40,[56][57][58], which can be concatenated with a text-driven articulatory predictor to achieve articulatory control over synthetic speech [56]. The second category is to build a unified model of acoustic and articulatory features for speech synthesis by describing the relationship between these two features in a context-dependent way [20,[59][60][61].…”

Section: Integrating Articulatory Features Into Spssmentioning

confidence: 99%

“…Some applications of a GMM-based articulatory-toacoustic mapping were presented in [57,58]. In [57], the GMM-based articulatory-to-acoustic mapping was combined with a GMM-based acoustic-to-articulatory inversion mapping and articulatory trajectory manipulation to achieve articulatory control over recorded natural speech.…”

Section: Articulatory-to-acoustic Mappingmentioning

confidence: 99%

The use of articulatory movement data in speech synthesis applications: An overview — Application of articulatory movements using machine learning algorithms —

Richmond

Ling

Yamagishi

2015

Acoust. Sci. & Tech.

View full text Add to dashboard Cite

This paper describes speech processing work in which articulator movements are used in conjunction with the acoustic speech signal and/or linguistic information. By ''articulator movements,'' we mean the changing positions of human speech articulators such as the tongue and lips, which may be recorded by electromagnetic articulography (EMA), amongst other articulography techniques. Specifically, we provide an overview of: i) inversion mapping techniques, where we estimate articulator movements from a given new speech waveform automatically; ii) statistical voice conversion and speech synthesis techniques which use articulator movements as part of the process to generate synthetic speech, and also make it intuitively controllable via articulation; and iii) automatic prediction (or synthesis) of articulator movements from any given new text input.

show abstract

Articulatory Controllable Speech Modification Based on Statistical Inversion and Production Mappings

Cited by 10 publications

References 43 publications

sprocket: Open-Source Voice Conversion Software

sprocket: Open-Source Voice Conversion Software

NU Voice Conversion System for the Voice Conversion Challenge 2018

The use of articulatory movement data in speech synthesis applications: An overview — Application of articulatory movements using machine learning algorithms —

Contact Info

Product

Resources

About