Channel-Recurrent Autoencoding for Image Modeling

Shang, Wenling; Sohn, Kihyuk; Tian, Yuandong

doi:10.1109/wacv.2018.00136

Cited by 10 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…VAE-GAN hybrids [76][77][78][79][80] function by combining a VAE's encoder into the GANs generator network, creating an unsupervised generative model capable of encoding, generating, and discriminating image samples. The goal of the combined network is to improve image sample quality, representation learning, sample diversity, and training stability compared to individual VAE and GAN models.…”

Section: Generative Adversarial Network Variational Autoencoders and ...mentioning

confidence: 99%

A Review of Synthetic Image Data and Its Use in Computer Vision

Man

Chahl

2022

J. Imaging

View full text Add to dashboard Cite

Development of computer vision algorithms using convolutional neural networks and deep learning has necessitated ever greater amounts of annotated and labelled data to produce high performance models. Large, public data sets have been instrumental in pushing forward computer vision by providing the data necessary for training. However, many computer vision applications cannot rely on general image data provided in the available public datasets to train models, instead requiring labelled image data that is not readily available in the public domain on a large scale. At the same time, acquiring such data from the real world can be difficult, costly to obtain, and manual labour intensive to label in large quantities. Because of this, synthetic image data has been pushed to the forefront as a potentially faster and cheaper alternative to collecting and annotating real data. This review provides general overview of types of synthetic image data, as categorised by synthesised output, common methods of synthesising different types of image data, existing applications and logical extensions, performance of synthetic image data in different applications and the associated difficulties in assessing data performance, and areas for further research.

show abstract

Section: Generative Adversarial Network Variational Autoencoders and ...mentioning

confidence: 99%

A Review of Synthetic Image Data and Its Use in Computer Vision

Man

Chahl

2022

J. Imaging

View full text Add to dashboard Cite

show abstract

“…Similarly to the VAE models for text/dialogue generation mentioned above, VAEs for image modeling generally apply many-to-one encoding and one-to-many decoding: They encode a whole image (that is a large number of pixels) in a single low-dimensional latent vector. Also they rely on knowledge accumulated on the use of deep neural networks for image processing, e.g., the decomposition of an image into successive feature maps using successive convolution layers and corresponding recomposition process, possibly combined with a multi-level or hierarchical latent encoding (Gulrajani et al, 2016;Shang et al, 2018). This makes those models a bit aside the temporal models we focus on, even if, as for text/dialogue generation VAEs, some propositions made in the literature on VAEs for image generation can be exploited in a more general framework.…”

Section: Aim Of the Papermentioning

confidence: 99%

Dynamical Variational Autoencoders: A Comprehensive Review

Girin,

Leglaive,

Bie

et al. 2020

Preprint

View full text Add to dashboard Cite

The Variational Autoencoder (VAE) is a powerful deep generative model that is now extensively used to represent high-dimensional complex data via a lowdimensional latent space that is learned in an unsupervised manner. In the original VAE model, input data vectors are processed independently. In the recent years, a series of papers have presented different extensions of the VAE to sequential data, that not only model the latent space, but also model the temporal dependencies within a sequence of data vectors and/or corresponding latent vectors, relying on recurrent neural networks or state space models. In this paper we perform an extensive literature review of these models. Importantly, we introduce and discuss a general class of models called Dynamical Variational Autoencoders (DVAEs) that encompass a large subset of these temporal VAE extensions. Then we present in details seven different instances of DVAE that were recently proposed in the literature, with an effort to homogenize the notations and presentation lines, as well as to relate those models with existing classical temporal models (that are also presented for the sake of completeness). We reimplemented those seven DVAE models and we present the results of an experimental benchmark that we conducted on the speech analysis-resynthesis task (the PyTorch code will be made publicly available). An extensive discussion is presented at the end of the paper, aiming to comment on important issues concerning the DVAE class of models and to describe future research guidelines.

show abstract

“…Besides, Sohn et al [36] also propose cVAE, whose encoder and decoder networks, totally different from standard VAE, are thoroughly convolutional. However, this kind of convolutional architecture (Figure 1(b)) produce disordered samples due to the latent representation sampled from spatially independent prior ignores the global structure of input images [37].…”

Section: A Analysis Of Encoder-decodr Architecturementioning

confidence: 99%

“…Consequently, we can find that the encoder-decoder architecture of these methods [13]- [15] proposed for facial attribute modification task is unsuitable for generating realistic new faces. Conversely, the latent representation of double-task models [18]- [20] may lack the capacity to model the complex data distribution [37], resulting in unexpected changes in modified images.…”

Section: A Analysis Of Encoder-decodr Architecturementioning

confidence: 99%

Adversarially Regularized U-Net-based GANs for Facial Attribute Modification and Generation

Zhang

Liu

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Modifying and generating facial images with desired attributes are two important and highly related tasks in the field of computer vision. Some current methods can take advantage of their relationship and use a unified model to handle them simultaneously. However, producing high visual quality images on both tasks is still a challenge. To tackle this issue, we propose a novel model called adversarially regularized U-net (ARU-net)-based generative adversarial networks (ARU-GANs). The ARU-net is the major part of the ARU-GAN and is inspired by the design principle of U-net. It uses skip connections to pass different-level features from encoder to decoder, which preserves sufficient attribute-independent details for the modification task. Besides, this U-net-like architecture employs an adversarial regularization term to guide the distribution of latent representation to match the prior distribution, which guarantees to generate meaningful faces from this prior. We also propose a joint training technique for the ARU-GAN, which enables the facial attribute modification and generation tasks to learn together during training. We perform experiments on celebfaces attributes (CelebA) dataset and make visual analysis and quantitative evaluation on both tasks, which demonstrates that our model can successfully produce high visual quality facial images. Also, the results show that learning two tasks jointly can lead to performance improvement compared with learning them individually. At last, we further validate the effectiveness of our method by making an ablation study and experimenting on another dataset. INDEX TERMS Generative adversarial networks, facial images, skip connections, adversarial regularization.

show abstract

Channel-Recurrent Autoencoding for Image Modeling

Cited by 10 publications

References 10 publications

A Review of Synthetic Image Data and Its Use in Computer Vision

A Review of Synthetic Image Data and Its Use in Computer Vision

Dynamical Variational Autoencoders: A Comprehensive Review

Adversarially Regularized U-Net-based GANs for Facial Attribute Modification and Generation

Contact Info

Product

Resources

About