Graphspeech: Syntax-Aware Graph Attention Network for Neural Speech Synthesis

Li, Rui; Şişman, Berrak; Li, Haizhou

doi:10.1109/icassp39728.2021.9413513

Cited by 14 publications

(8 citation statements)

References 27 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The frame-by-frame encoding approach encodes each speech frame to a vector. Various methods are used to extract the features of speech frames, such as Convolutional Layer, LSTM, Transformer [16], or build a graph in speech frames to get more detailed features [17]. Then the encoded speech frames can be used in many tasks, such as speech recognition [18] and direct speech translation [19].…”

Section: Related Workmentioning

confidence: 99%

A New Speech Encoder Based on Dynamic Framing Approach

Liu¹,

Yang²,

Zhou³

2023

Computer Modeling in Engineering &Amp; Sciences

View full text Add to dashboard Cite

Latent information is difficult to get from the text in speech synthesis. Studies show that features from speech can get more information to help text encoding. In the field of speech encoding, a lot of work has been conducted on two aspects. The first aspect is to encode speech frame by frame. The second aspect is to encode the whole speech to a vector. But the scale in these aspects is fixed. So, encoding speech with an adjustable scale for more latent information is worthy of investigation. But current alignment approaches only support frame-by-frame encoding and speech-to-vector encoding. It remains a challenge to propose a new alignment approach to support adjustable scale speech encoding. This paper presents the dynamic speech encoder with a new alignment approach in conjunction with frame-by-frame encoding and speech-to-vector encoding. The speech feature from our model achieves three functions. First, the speech feature can reconstruct the origin speech while the length of the speech feature is equal to the text length. Second, our model can get text embedding from speech, and the encoded speech feature is similar to the text embedding result. Finally, it can transfer the style of synthesis speech and make it more similar to the given reference speech.

show abstract

Section: Related Workmentioning

confidence: 99%

A New Speech Encoder Based on Dynamic Framing Approach

Liu¹,

Yang²,

Zhou³

2023

Computer Modeling in Engineering &Amp; Sciences

View full text Add to dashboard Cite

show abstract

“…In addition, the generation model benefits NAR neural TTS, constructing a NAR TTS utilizing a deep variational autoencoder with a residual attention mechanism that subtly refines the text-to-sound alignment (Liu et al 2021). Excessive smoothing is a severe problem for the NAR TTS model, which harms the performance of the NAR TTS model (Ren et al 2022).…”

Section: Non-autoregressive Neural Ttsmentioning

confidence: 99%

“…Traditional neural TTS models usually use phoneme sequences as input which haven't fully utilized the contextual semantic information of the target sentence. Therefore, many works improve the expressiveness of TTS by introducing syntactic information Liu, Sisman, and Li 2021), those methods by explicitly associating input phoneme embedding with syntactic relations. A word-level semantic representation method is proposed in (Zhou et al 2021) which is based on dependent structure and pre-trained BERT.…”

Section: Introductionmentioning

confidence: 99%

SeDepTTS: Enhancing the Naturalness via Semantic Dependency and Local Convolution for Text-to-Speech Synthesis

Jiang

Gao

et al. 2023

AAAI

View full text Add to dashboard Cite

Self-attention-based networks have obtained impressive performance in parallel training and global context modeling. However, it is weak in local dependency capturing, especially for data with strong local correlations such as utterances. Therefore, we will mine linguistic information of the original text based on a semantic dependency and the semantic relationship between nodes is regarded as prior knowledge to revise the distribution of self-attention. On the other hand, given the strong correlation between input characters, we introduce a one-dimensional (1-D) convolution neural network (CNN) producing query(Q) and value(V) in the self-attention mechanism for a better fusion of local contextual information. Then, we migrate this variant of the self-attention networks to speech synthesis tasks and propose a non-autoregressive (NAR) neural Text-to-Speech (TTS): SeDepTTS. Experimental results show that our model yields good performance in speech synthesis. Specifically, the proposed method yields significant improvement for the processing of pause, stress, and intonation in speech.

show abstract

“…Results showed the effectiveness of so-called attentional GNNs in transferring the metric representation learned from training classes to novel classes. Liu et al [30] demonstrated the application of GNNs to neural speech synthesis, for which they are used to encode explicitly the syntactic relationship of the different elements within a sentence. Jung et al [31] showed how GATs can be used to learn utterance-level relationships between speakers and how a GAT architecture with residual connections can be adapted to compute utterance-level similarity scores for speaker verification.…”

Section: Related Workmentioning

confidence: 99%

Graph Attention Networks for Anti-Spoofing

Tak¹,

Jung²,

Patino³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

The cues needed to detect spoofing attacks against automatic speaker verification are often located in specific spectral subbands or temporal segments. Previous works show the potential to learn these using either spectral or temporal selfattention mechanisms but not the relationships between neighbouring sub-bands or segments. This paper reports our use of graph attention networks (GATs) to model these relationships and to improve spoofing detection performance. GATs leverage a self-attention mechanism over graph structured data to model the data manifold and the relationships between nodes. Our graph is constructed from representations produced by a ResNet. Nodes in the graph represent information either in specific sub-bands or temporal segments. Experiments performed on the ASVspoof 2019 logical access database show that our GAT-based model with temporal attention outperforms all of our baseline single systems. Furthermore, GAT-based systems are complementary to a set of existing systems. The fusion of GAT-based models with more conventional countermeasures delivers a 47% relative improvement in performance compared to the best performing single GAT system.

show abstract

Graphspeech: Syntax-Aware Graph Attention Network for Neural Speech Synthesis

Cited by 14 publications

References 27 publications

A New Speech Encoder Based on Dynamic Framing Approach

A New Speech Encoder Based on Dynamic Framing Approach

SeDepTTS: Enhancing the Naturalness via Semantic Dependency and Local Convolution for Text-to-Speech Synthesis

Graph Attention Networks for Anti-Spoofing

Contact Info

Product

Resources

About