On the localness modeling for the self-attention based end-to-end speech synthesis

Yang, Shan; Lu, Hanqing; Kang, Shiyin; Xue, Liumeng; Xiao, Jin‐Xin; Su, Dan; Xie, Lei; Yu, Dong

doi:10.1016/j.neunet.2020.01.034

Cited by 42 publications

(22 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the weights theoretically can distribute over all input positions, they are often concentrated locally, particularly with output position i focusing on input position i. Previous works on various sequence tasks (Yang et al, 2020;Zhang et al, 2020b) have shown heavy weights on the diagonal of the encoder selfattention matrices.…”

Section: Position-based Self-attention Querymentioning

confidence: 99%

Improving Zero-Shot Translation by Disentangling Positional Information

Liu

Niehues

Cross

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Multilingual neural machine translation has shown the capability of directly translating between language pairs unseen in training, i.e. zero-shot translation. Despite being conceptually attractive, it often suffers from low output quality. The difficulty of generalizing to new translation directions suggests the model representations are highly specific to those language pairs seen in training. We demonstrate that a main factor causing the language-specific representations is the positional correspondence to input tokens. We show that this can be easily alleviated by removing residual connections in an encoder layer. With this modification, we gain up to 18.5 BLEU points on zero-shot translation while retaining quality on supervised directions. The improvements are particularly prominent between related languages, where our proposed model outperforms pivot-based translation. Moreover, our approach allows easy integration of new languages, which substantially expands translation coverage. By thorough inspections of the hidden layer outputs, we show that our approach indeed leads to more languageindependent representations. 1

show abstract

Section: Position-based Self-attention Querymentioning

confidence: 99%

Improving Zero-Shot Translation by Disentangling Positional Information

Liu

Niehues

Cross

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…Recurrent neural networks (RNN) have been proved to excel at various sequential tasks, such as speech recognition [79], speech synthesis [80], handwriting recognition [81], and image to text [82]. Particularly, Long Short-Term Memory (LSTM) layers [83], transformers and self-attention mechanism [84] are the robust architecture for modelling long range sequence data with auto correlations like time series data, natural languages etc.…”

Section: Figure 1: Distribution Of Different Type Of Datasets (A) Datmentioning

confidence: 99%

A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

Sampath

Maurtua

Martín

et al. 2020

Preprint

View full text Add to dashboard Cite

Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets.In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

show abstract

“…Recurrent neural networks (RNN) have been proved to excel at various sequential tasks, such as speech recognition [80], speech synthesis [81], handwriting recognition [82], and image to text [83]. Particularly, Long Short-Term Memory (LSTM) layers [84], transformers and self-attention mechanism [85] Van den Oord et al [69] designed two variants of recurrent image models: PixelRNN and PixelCNN.…”

Section: Introductionmentioning

confidence: 99%

A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

Sampath

Maurtua

Martín

et al. 2021

Preprint

View full text Add to dashboard Cite

Any computer vision application development starts off by acquiring images and data, then preprocessing and pattern recognition steps to perform a task. When the acquired images are highly imbalanced and not adequate, the desired task may not be achievable. Unfortunately, the occurrence of imbalance problems in acquired image datasets in certain complex real-world problems such as anomaly detection, emotion recognition, medical image analysis, fraud detection, metallic surface defect detection, disaster prediction, etc., are inevitable. The performance of computer vision algorithms can significantly deteriorate when the training dataset is imbalanced. In recent years, Generative Adversarial Neural Networks (GANs) have gained immense attention by researchers across a variety of application domains due to their capability to model complex real-world image data. It is particularly important that GANs can not only be used to generate synthetic images, but also its fascinating adversarial learning idea showed good potential in restoring balance in imbalanced datasets.In this paper, we examine the most recent developments of GANs based techniques for addressing imbalance problems in image data. The real-world challenges and implementations of synthetic image generation based on GANs are extensively covered in this survey. Our survey first introduces various imbalance problems in computer vision tasks and its existing solutions, and then examines key concepts such as deep generative image models and GANs. After that, we propose a taxonomy to summarize GANs based techniques for addressing imbalance problems in computer vision tasks into three major categories: 1. Image level imbalances in classification, 2. object level imbalances in object detection and 3. pixel level imbalances in segmentation tasks. We elaborate the imbalance problems of each group, and provide GANs based solutions in each group. Readers will understand how GANs based techniques can handle the problem of imbalances and boost performance of the computer vision algorithms.

show abstract

On the localness modeling for the self-attention based end-to-end speech synthesis

Cited by 42 publications

References 11 publications

Improving Zero-Shot Translation by Disentangling Positional Information

Improving Zero-Shot Translation by Disentangling Positional Information

A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

A Survey on Generative Adversarial Networks for imbalance problems in computer vision tasks

Contact Info

Product

Resources

About