GraphTTS: Graph-to-Sequence Modelling in Neural Text-to-Speech

Sun, Aolan; Wang, Jianzong; Cheng, Ning; Peng, Huayi; Zeng, Zhen; Xiao, Jing

doi:10.1109/icassp40776.2020.9053355

Cited by 13 publications

(9 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The relational gated graph network (RGGN) aims to learn the semantic representation from the dependency graph in section 2.1. The gated graph neural network (GGNN) is adopted due to its long-term propagation of information flow [23]. Additionally, different weight matrices corresponding to different types of edges are introduced to GGNN inspired by relational graph convolutional network [26].…”

Section: Relational Gated Graph Networkmentioning

confidence: 99%

“…However, Graph-Speech uses a recurrent neural network (RNN) based relation encoder to model the dependency relation, which is insufficient to capture information considering the complex sentence structure. Graph neural network (GNN) [21,22] is further introduced to TTS for its ability to learn the representation via message passing among the nodes of graphs, like GraphTTS [23] and GraphPB [24], but all of them use simple structures designed only from the text and without considering deeper semantics.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis

Zhou¹,

Song²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Semantic information of a sentence is crucial for improving the expressiveness of a text-to-speech (TTS) system, but can not be well learned from the limited training TTS dataset just by virtue of the nowadays encoder structures. As large scale pretrained text representation develops, bidirectional encoder representations from transformers (BERT) has been proven to embody text-context semantic information and applied to TTS as additional input. However BERT can not explicitly associate semantic tokens from point of dependency relations in a sentence. In this paper, to enhance expressiveness, we propose a semantic representation learning method based on graph neural network, considering dependency relations of a sentence. Dependency graph of input text is composed of edges from dependency tree structure considering both the forward and the reverse directions. Semantic representations are then extracted at word level by the relational gated graph network (RGGN) fed with features from BERT as nodes input. Upsampled semantic representations and character-level embeddings are concatenated to serve as the encoder input of Tacotron-2. Experimental results show that our proposed method outperforms the baseline using vanilla BERT features both in LJSpeech and Blizzard Challenge 2013 datasets, and semantic representations learned from the reverse direction are more effective for enhancing expressiveness.

show abstract

Section: Relational Gated Graph Networkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis

Zhou¹,

Song²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Sequence-to-Sequence learning is a part of ML and a method of neural networks that is mostly utilized in language processing models [17][18][19][20][21][22][23]. It can be implemented with recurrent neural networks (RNNs) using encoder-decoder based machine interpretation that maps an input sequence to a yield of a succession of output sequence with a tag and consideration esteem.…”

Section: Introductionmentioning

confidence: 99%

A systematic review on sequence-to-sequence learning with neural network and its models

Yousuf

Lahzi

Salloum

et al. 2021

IJECE

View full text Add to dashboard Cite

We develop a precise writing survey on sequence-to-sequence learning with neural network and its models. The primary aim of this report is to enhance the knowledge of the sequence-to-sequence neural network and to locate the best way to deal with executing it. Three models are mostly used in sequence-to-sequence neural network applications, namely: recurrent neural networks (RNN), connectionist temporal classification (CTC), and attention model. The evidence we adopted in conducting this survey included utilizing the examination inquiries or research questions to determine keywords, which were used to search for bits of peer-reviewed papers, articles, or books at scholastic directories. Through introductory hunts, 790 papers, and scholarly works were found, and with the assistance of choice criteria and PRISMA methodology, the number of papers reviewed decreased to 16. Every one of the 16 articles was categorized by their contribution to each examination question, and they were broken down. At last, the examination papers experienced a quality appraisal where the subsequent range was from 83.3% to 100%. The proposed systematic review enabled us to collect, evaluate, analyze, and explore different approaches of implementing sequence-to-sequence neural network models and pointed out the most common use in machine learning. We followed a methodology that shows the potential of applying these models to real-world applications.

show abstract

“…Another benefit of SAN is to function with intra-attention [14,16] , which has a shorter path to model long distance context. Despite the progress [15], Transformer TTS doesn't explicitly associate input text with output utterances from syntactic point of view at sentence level, which is proven useful in speaking style and prosody modeling [17][18][19][20][21]. As a result, the rendering of utterance is adversely affected especially for long sentences.…”

Section: Introductionmentioning

confidence: 99%

Graphspeech: Syntax-Aware Graph Attention Network for Neural Speech Synthesis

Şişman

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Attention-based end-to-end text-to-speech synthesis (TTS) is superior to conventional statistical methods in many ways. Transformer-based TTS is one of such successful implementations. While Transformer TTS models the speech frame sequence well with a self-attention mechanism, it does not associate input text with output utterances from a syntactic point of view at sentence level. We propose a novel neural TTS model, denoted as GraphSpeech, that is formulated under graph neural network framework. GraphSpeech encodes explicitly the syntactic relation of input lexical tokens in a sentence, and incorporates such information to derive syntactically motivated character embeddings for TTS attention mechanism. Experiments show that GraphSpeech consistently outperforms the Transformer TTS baseline in terms of spectrum and prosody rendering of utterances.

show abstract

GraphTTS: Graph-to-Sequence Modelling in Neural Text-to-Speech

Cited by 13 publications

References 12 publications

Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis

Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis

A systematic review on sequence-to-sequence learning with neural network and its models

Graphspeech: Syntax-Aware Graph Attention Network for Neural Speech Synthesis

Contact Info

Product

Resources

About