Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis

Zhou, Yong‐Gui; Song, Changhe; Li, Jingbei; Wu, Zhiyong; Bian, Yanyao; Su, Dan; Meng, Helen

doi:10.21437/interspeech.2022-10061

Cited by 5 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, part of synthetic speech samples 5 are disclosed in SAG-Tacotron, which makes the comparison more credible. • BERT-Dep: This model is based on dependent structure and pre-trained BERT to generate word-level semantic representation information, which is fused into the latent representations of Tacotron2 as additional features (Zhou et al 2021). It is the latest based on semantic information to improve the naturalness and expressiveness of speech synthesis 6 .…”

Section: Baseline Methodsmentioning

confidence: 99%

“…Therefore, many works improve the expressiveness of TTS by introducing syntactic information Liu, Sisman, and Li 2021), those methods by explicitly associating input phoneme embedding with syntactic relations. A word-level semantic representation method is proposed in (Zhou et al 2021) which is based on dependent structure and pre-trained BERT. Although the above works explore the validity of linguistic knowledge, the local modeling performance has not improved significantly on the TTS task.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SeDepTTS: Enhancing the Naturalness via Semantic Dependency and Local Convolution for Text-to-Speech Synthesis

Jiang

Gao

et al. 2023

AAAI

View full text Add to dashboard Cite

Self-attention-based networks have obtained impressive performance in parallel training and global context modeling. However, it is weak in local dependency capturing, especially for data with strong local correlations such as utterances. Therefore, we will mine linguistic information of the original text based on a semantic dependency and the semantic relationship between nodes is regarded as prior knowledge to revise the distribution of self-attention. On the other hand, given the strong correlation between input characters, we introduce a one-dimensional (1-D) convolution neural network (CNN) producing query(Q) and value(V) in the self-attention mechanism for a better fusion of local contextual information. Then, we migrate this variant of the self-attention networks to speech synthesis tasks and propose a non-autoregressive (NAR) neural Text-to-Speech (TTS): SeDepTTS. Experimental results show that our model yields good performance in speech synthesis. Specifically, the proposed method yields significant improvement for the processing of pause, stress, and intonation in speech.

show abstract

Section: Baseline Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

SeDepTTS: Enhancing the Naturalness via Semantic Dependency and Local Convolution for Text-to-Speech Synthesis

Jiang

Gao

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…Kim, Kong, and Son (2021) Tatanov, Beliaev, and Ginsburg (2022) boosted the expressiveness of speech by applying various methods proposed in the field of natural language processing (NLP) to the speech domain. Especially, GraphSpeech (Liu, Sisman, and Li 2021) and Relational Gated Graph Network (RGGN) (Zhou et al 2022) claimed the syntactic and semantic information of text affects the naturalness and expressiveness of speech. They improved the performance by utilizing graph networks focused on the representation based on dependency relations.…”

Section: Introductionmentioning

confidence: 99%

“…We need the proper way to align the dependency relations with H S because we use them simultaneously. Therefore, we utilize Word-level Average Pooling to align between them, similar to Subword-to-Word Mapping by Zhou et al (2022). We use average pooling (AP ) based on word level represented as,…”

Section: Introductionmentioning

confidence: 99%

“…Semantic-level Relation Encoding Previous work (Zhou et al 2022) proposes RGGN utilizing the graph networks based on the dependency tree. Since the feature of the dependency tree is related to the prosody of speech (Köhn, Baumann, and Dörfler 2018;Liu, Sisman, and Li 2021;Zhou et al 2022) and graph networks are suitable for encoding tree structures, they report improved results compared to baseline. However, they only use graph networks to encode neighbor words in the tree and don't seem to consider linguistic features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

RWEN-TTS: Relation-Aware Word Encoding Network for Natural Text-to-Speech Synthesis

Oh¹,

Noh²,

Hong³

et al. 2023

AAAI

View full text Add to dashboard Cite

With the advent of deep learning, a huge number of text-to-speech (TTS) models which produce human-like speech have emerged. Recently, by introducing syntactic and semantic information w.r.t the input text, various approaches have been proposed to enrich the naturalness and expressiveness of TTS models. Although these strategies showed impressive results, they still have some limitations in utilizing language information. First, most approaches only use graph networks to utilize syntactic and semantic information without considering linguistic features. Second, most previous works do not explicitly consider adjacent words when encoding syntactic and semantic information, even though it is obvious that adjacent words are usually meaningful when encoding the current word. To address these issues, we propose Relation-aware Word Encoding Network (RWEN), which effectively allows syntactic and semantic information based on two modules (i.e., Semantic-level Relation Encoding and Adjacent Word Relation Encoding). Experimental results show substantial improvements compared to previous works.

show abstract

Semantic dependency and local convolution for enhancing naturalness and tone in text-to-speech synthesis

Jiang,

Gao,

et al. 2024

Neurocomputing

View full text Add to dashboard Cite

Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis

Cited by 5 publications

References 0 publications

SeDepTTS: Enhancing the Naturalness via Semantic Dependency and Local Convolution for Text-to-Speech Synthesis

SeDepTTS: Enhancing the Naturalness via Semantic Dependency and Local Convolution for Text-to-Speech Synthesis

RWEN-TTS: Relation-Aware Word Encoding Network for Natural Text-to-Speech Synthesis

Semantic dependency and local convolution for enhancing naturalness and tone in text-to-speech synthesis

Contact Info

Product

Resources

About