Attention-Based Speaker Embeddings for One-Shot Voice Conversion

Ishihara, Tatsuma; Saito, Daisuke

doi:10.21437/interspeech.2020-2512

Cited by 9 publications

(12 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To obtain phoneme-level style information, the key idea is an attention mechanism [21] relating style to content [22,23]. Our approach similarly assumes that the style information is related to the content, so instead of only using a fixed-length vector to represent the style of the whole utterance, the style information should rely on the content and change with time.…”

Section: Style To Phoneme Attentionmentioning

confidence: 99%

Two-Pathway Style Embedding for Arbitrary Voice Conversion

Shi²,

Chen

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

Arbitrary voice conversion, also referred to as zero-shot voice conversion, has recently attracted increased attention in the literature. Although disentangling the linguistic and style representations for acoustic features is an effective way to achieve zero-shot voice conversion, the problem of how to convert to a natural speaker style is challenging because of the intrinsic variabilities of speech and the difficulties of completely decoupling them. For this reason, in this paper, we propose a Two-Pathway Style Embedding Voice Conversion framework (TPSE-VC) for realistic and natural speech conversion. The novel feature of this method is to simultaneously embed sentence-level and phoneme-level style information. A novel attention mechanism is proposed to implement the implicit alignment for timbre style and phoneme content, further embedding a phoneme-level style representation. In addition, we consider embedding the complete set of time steps of audio style into a fixed-length vector to obtain the sentence-level style representation. Moreover, TPSE-VC does not require any pre-trained models, and is only trained with non-parallel speech data. Experimental results demonstrate that the proposed TPSE-VC outperforms the state-of-theart results on zero-shot voice conversion.

show abstract

Section: Style To Phoneme Attentionmentioning

confidence: 99%

Two-Pathway Style Embedding for Arbitrary Voice Conversion

Shi²,

Chen

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

show abstract

“…If the performance of VC is not precise enough, voice augmentation for VC is possible. Different augmentations techniques for VC were proposed such as attention-based speaker embeddings for one-shot VC and data augmentationbased non-parallel VC [27]- [28].…”

Section: Introductionmentioning

confidence: 99%

Voice Conversion Based Augmentation and a Hybrid CNN-LSTM Model for Improving Speaker-Independent Keyword Recognition on Limited Datasets

Wubet

Lian

2022

IEEE Access

View full text Add to dashboard Cite

Keyword recognition is the basis of speech recognition, and its application is rapidly increasing in keyword spotting, robotics, and smart home surveillance. Because of these advanced applications, improving the accuracy of keyword recognition is crucial. In this paper, we proposed voice conversion (VC) -based augmentation to increase the limited training dataset and a fusion of a convolutional neural network (CNN) and long-short term memory (LSTM) model for robust speaker-independent isolated keyword recognition. Collecting and preparing a sufficient amount of voice data for speaker-independent speech recognition is a tedious and bulky task. To overcome this, we generated new raw voices from the original voices using an auxiliary classifier conditional variational autoencoder (ACVAE) method. In this study, the main intention of voice conversion is to obtain numerous and various human-like keywords' voices that are not identical to the source and target speakers' pronunciation. Parallel VC was used to accurately maintain the linguistic content. We examined the performance of the proposed voice conversion augmentation techniques using robust deep neural network algorithms. Original training data, excluding generated voice using other data augmentation and regularization techniques, were considered as the baseline. The results showed that incorporating voice conversion augmentation into the baseline augmentation techniques and applying the CNN-LSTM model improved the accuracy of isolated keyword recognition.

show abstract

“…Based on the U-net [29] structure, Li et al [30], Wu et al [24], and Li et al [28] extract utterancelevel speaker representations from multiple stacking layers and feed them to corresponding decoder layers. To access timevarying speaker information, some studies [25]- [27] extract variable-length speaker representation and fuse it into the converted speech according to the content-based alignment between source speech and target speaker speech.…”

Section: Introductionmentioning

confidence: 99%

“…Moreover, speech production research [31] shows that the frequency distributions of speech from different speakers lead to varying speaker timbre information in frequency channels [32]- [34]. Speech content, such as vowels, consonants, and para-linguistic features, carry distinct speaker timbre information reflected in temporal and frequency channel dimensions while silent speech segments apparently convey no speaker timbre information [27]. On the other hand, the human speech production mechanism is hierarchical [31], [35] in nature from long-term airflow generation to fine-grained phoneme-related articulator movements and vocal filtering.…”

Section: Introductionmentioning

confidence: 99%

JDE multi-object tracking algorithm integrating multi-level semantic enhancement

Wang

Wang²,

Li³

et al. 2022

西北工业大学学报

View full text Add to dashboard Cite

In order to solve the problem of target ID switching caused by target occlusion and insufficient ID information and location information extraction in JDE(joint detection and embedding) algorithm, an improved multi-target tracking algorithm based on JDE is proposed in this paper. Firstly, the SPA feature space pyramid attention module is used to expand the receptive field and obtain more abundant semantic information to improve the detection accuracy of the model for different scale targets. Secondly, the FCN network makes the header and ID Embedding task collaborative learning to alleviate the excessive competition and enhance the original semantic information, effectively reducing the number of ID switching. Finally, PCCs-Ma motion measurement can strengthen the connection between Kalman filtering prediction and observation, and improve the reliability of similarity discrimination of motion characteristics. In order to verify the effectiveness of the algorithm, the JDE algorithm and the proposed algorithm are compared in the same experimental environment. The experimental results show that the average accuracy of model detection is improved by 3.94 %. On the MOT16 dataset, the MOTA and IDF1 indexes are increased by 6.9 %, and the number of ID switching of the improved algorithm is significantly reduced, and good tracking results are achieved.

show abstract

Attention-Based Speaker Embeddings for One-Shot Voice Conversion

Cited by 9 publications

References 12 publications

Two-Pathway Style Embedding for Arbitrary Voice Conversion

Two-Pathway Style Embedding for Arbitrary Voice Conversion

Voice Conversion Based Augmentation and a Hybrid CNN-LSTM Model for Improving Speaker-Independent Keyword Recognition on Limited Datasets

JDE multi-object tracking algorithm integrating multi-level semantic enhancement

Contact Info

Product

Resources

About