Graph2Pix: A Graph-Based Image to Image Translation Framework

Gokay, Dilara; Simsar, Enis; Atici, Efehan; Ahmetoğlu, Alper; Yüksel, Atıf Emre; Yanardag, Pinar

doi:10.1109/iccvw54120.2021.00227

Cited by 3 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, Park et al [24] proposed an approach based on patchwise contrastive learning and adversarial learning, while [25] explored a hierarchical tree structure to organize labels and a new translation process. A peculiar approach was developed by [26] which exploited a rich dataset collected through Artbreeder [27] to output a single image from a graph-like structure. Finally, Dai et al [28] learned a sequence of invertible mappings which led to a flow-version of popular GANs, such as StarGAN, AGGAN and CyCADA, with similar performances but half of the training parameters.…”

Section: Related Workmentioning

confidence: 99%

aRTIC GAN: A Recursive Text-Image-Conditioned GAN

et al. 2022

View full text Add to dashboard Cite

Generative Adversarial Networks have recently demonstrated the capability to synthesize photo-realistic real-world images. However, they still struggle to offer high controllability of the output image, even if several constraints are provided as input. In this work, we present a Recursive Text-Image-Conditioned GAN (aRTIC GAN), a novel approach for multi-conditional image generation under concurrent spatial and text constraints. It employs few line drawings and short descriptions to provide informative yet human-friendly conditioning. The proposed scenario is based on accessible constraints with high degrees of freedom: sketches are easy to draw and add strong restrictions on the generated objects, such as their orientation or main physical characteristics. Text on its side is so common and expressive that easily enforces information otherwise impossible to provide with minimal illustrations, such as objects components color, color shades, etc. Our aRTIC GAN is suitable for the sequential generation of multiple objects due to its compact design. In fact, the algorithm exploits the previously generated image in conjunction with the sketch and the text caption, resulting in a recurrent approach. We developed three network blocks to tackle the fundamental problems of catching captions’ semantic meanings and of handling the trade-off between smoothing grid-pattern artifacts and visual detail preservation. Furthermore, a compact three-task discriminator (covering global, local and textual aspects) was developed to preserve a lightweight and robust architecture. Extensive experiments proved the validity of aRTIC GAN and show that the combined use of sketch and description allows us to avoid explicit object labeling.

show abstract

Section: Related Workmentioning

confidence: 99%

aRTIC GAN: A Recursive Text-Image-Conditioned GAN

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The methods presented in this field can be divided into two categories: GANs (i.e. the methods relying on generative adversarial networks) [2][3][4] and non-GANs [5]. GANs have been very successful in this field because of their great ability to learn the distribution of data [6].…”

Section: Introductionmentioning

confidence: 99%

ARDA‐UNIT recurrent dense self‐attention block with adaptive feature fusion for unpaired (unsupervised) image‐to‐image translation

Ghombavani

Fadaeieslam

2023

IET Image Processing

View full text Add to dashboard Cite

One of the most challenging topics in artificial intelligence is image‐to‐image translation, the purpose of which is generating images close to those in the target domain while preserving the important features of the images in the source domain. In this direction, various types of generative adversarial networks have been developed. ARDA‐UNIT, presented in this paper, seeks to meet the main challenges of these networks, that is, producing a high‐quality image in a reasonable amount of time, and transferring content between two images with different structures. The proposed recurrent dense self‐attention block, applied in ARDA‐UNIT's generator latent space, simultaneously increases its generating capability and decreases the training parameters. ARDA‐UNIT has a feature extraction module which feeds both the generator and the discriminator. This module uses a new adaptive feature fusion method which combines multi‐scale features in such a way that the characteristics of each scale are preserved. The module also uses a pre‐trained CNN that reduces the training parameters. Moreover, a feature similarity loss is introduced that guides the model to change the structure of the source domain in accordance with that in the target domain. Experiments performed on different datasets using FID, KID and IS evaluation criteria have shown that the model reduces computational loads, transfers structures well, and achieves better qualities.

show abstract

A Review of Synthetic Image Data and Its Use in Computer Vision

Man

Chahl

2022

J. Imaging

View full text Add to dashboard Cite

Development of computer vision algorithms using convolutional neural networks and deep learning has necessitated ever greater amounts of annotated and labelled data to produce high performance models. Large, public data sets have been instrumental in pushing forward computer vision by providing the data necessary for training. However, many computer vision applications cannot rely on general image data provided in the available public datasets to train models, instead requiring labelled image data that is not readily available in the public domain on a large scale. At the same time, acquiring such data from the real world can be difficult, costly to obtain, and manual labour intensive to label in large quantities. Because of this, synthetic image data has been pushed to the forefront as a potentially faster and cheaper alternative to collecting and annotating real data. This review provides general overview of types of synthetic image data, as categorised by synthesised output, common methods of synthesising different types of image data, existing applications and logical extensions, performance of synthetic image data in different applications and the associated difficulties in assessing data performance, and areas for further research.

show abstract

Graph2Pix: A Graph-Based Image to Image Translation Framework

Cited by 3 publications

References 14 publications

aRTIC GAN: A Recursive Text-Image-Conditioned GAN

aRTIC GAN: A Recursive Text-Image-Conditioned GAN

ARDA‐UNIT recurrent dense self‐attention block with adaptive feature fusion for unpaired (unsupervised) image‐to‐image translation

A Review of Synthetic Image Data and Its Use in Computer Vision

Contact Info

Product

Resources

About