SoloGAN: Multi-domain Multimodal Unpaired Image-to-Image Translation via a Single Generative Adversarial Network

Huang, S.; He, Cheng; Cheng, Ran

doi:10.1109/tai.2022.3187384

Cited by 8 publications

(4 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Image-to-image (I2I) translation aims to learn the mapping between different visual domains. Generative Adversarial Networks (GAN) have become the ubiquitous choice for such computer vision task [16] [17] [18]. One artifact that has proven to improve translation is the cycle consistency loss, where the generated image is used as source for another translation and the rror is incorporated in the loss.…”

Section: Related Work A: Shared Representation Of Sensor Modalitiesmentioning

confidence: 99%

Image Translation and Reconstruction Using a Single Dual Mode Lightweight Encoder

Amendola,

Cenkeramaddi,

Jha

2024

IEEE Access

View full text Add to dashboard Cite

The richness of textures and semantic information from RGB images can be supplemented in computer vision by the robustness of thermal images to light variations and weather artifacts. While many models rely on inputs from one sensor modality, image translation among modalities can be a solution. The existing works use large models that only work in one translation direction. This cause problems in limited computation applications, as well as a lack of flexibility to work interchangeably for different modalities. Three channel cameras extract visually rich features, but processing them on embedded platforms becomes a bottleneck. Furthermore, edge computing systems impose the burden of compressing data to be sent elsewhere. To address these issues, we propose a novel architecture with a single lightweight encoder capable of working in dual mode, encoding inputs from both grayscale an thermal images into very compact latent vectors. The encoding is then used for cross-modal image translation, grayscale image colorization and thermal image reconstruction, thus allowing (a) different downstream tasks on different modalities, (b) visually rich features from grayscale images and (c) data compression. Four different generators are employed and the training occurs in adversarial fashion with two discriminators. The loss function proposed contains not only adversarial terms but also reconstruction error terms. They induce consistency and contrast preservation across translation and reconstruction. The results backed by evaluation over multiple metrics demonstrate that the model performs the tasks with competitive quality of translation/reconstruction of images with different lighting conditions. Finally, we perform ablation studies to demonstrate the effectiveness of loss terms combined. INDEX TERMS Generative Adversarial Networks (GAN), Image reconstruction, Image-to-image (I2I) translation

show abstract

Section: Related Work A: Shared Representation Of Sensor Modalitiesmentioning

confidence: 99%

Image Translation and Reconstruction Using a Single Dual Mode Lightweight Encoder

Amendola,

Cenkeramaddi,

Jha

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…extending our method to multi-domain image translation by applying domain classification loss [9] and similar techniques [63][64][65].…”

Section: Plos Onementioning

confidence: 99%

Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module

Oh,

Noh

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

Recently, contrastive learning has gained popularity in the field of unsupervised image-to-image (I2I) translation. In a previous study, a query-selected attention (QS-Attn) module, which employed an attention matrix with a probability distribution, was used to maximize the mutual information between the source and translated images. This module selected significant queries using an entropy metric computed from the attention matrix. However, it often selected many queries with equal significance measures, leading to an excessive focus on the background. In this study, we proposed a dual-learning framework with QS-Attn and convolutional block attention module (CBAM) called object-stable dual contrastive learning generative adversarial network (OS-DCLGAN). In this paper, we utilize a CBAM, which learns what and where to emphasize or suppress, thereby refining intermediate features effectively. This CBAM was integrated before the QS-Attn module to capture significant domain information for I2I translation tasks. The proposed framework outperformed recently introduced approaches in various I2I translation tasks, showing its effectiveness and versatility. The code is available at https://github.com/RedPotatoChip/OSUDL

show abstract

“…where T ≡ T (α 0 , C L , C L D , C P , h) represents the joint distribution of target performances, t represents an arbitrary instance of T , and Q represents the joint distribution of design variables q. Various generators, such as Variational Autoencoders (VAEs) [26], Generative Adversarial Networks (GANs) [27] Conditional Generative Adversarial Networks (CGANs) [28], and SoloGAN [29], have been introduced in previous studies to model this Probability Density Function (PDF). However, the target aerodynamic performance space spanned by T in the training dataset for this model is often limited and discrete, leading to an uneven distribution.…”

Section: Generatormentioning

confidence: 99%

Evolutionary generative design of supercritical airfoils: an automated approach driven by small data

Sun,

Wang,

Cheng

et al. 2023

Complex Intell. Syst.

Self Cite

View full text Add to dashboard Cite

Supercritical airfoils are critical components in the design of commercial wide-body aircraft wings due to their ability to enhance aerodynamic performance in transonic flow regimes. However, traditional design methods for supercritical airfoils can be time-consuming and require significant manual effort, not to mention the high cost associated with computational fluid dynamics analysis. To address these challenges, this paper introduces a highly automated approach for supercritical airfoil design, called Evolutionary Generative Design (EvoGD). The EvoGD approach is based on the framework of Evolutionary Computation and employs a series of sophisticated data-driven generative models incorporated with physical information to iteratively refine initial airfoil shapes, resulting in improved aerodynamic performances and reduced constraint violations. Moreover, to speed up the evaluation of the generated airfoils, a series of accurate and efficient data-driven predictors are utilized. The efficacy of the EvoGD approach was demonstrated through experiments on a dataset of 501 supercritical airfoils, including one baseline design and 500 randomly perturbed airfoils. On average, the generated airfoils showed improved performance in terms of buffet lift coefficient, cruise lift-to-drag ratio, and thickness by 5%, 4%, and 1%, respectively. The best generated airfoil outperformed the baseline design in terms of critical buffet lift coefficient and cruise lift-to-drag ratio by 7.1% and 6.4%, respectively. The entire design process was completed in less than an hour on a personal computer, highlighting the high efficiency and scalability of the EvoGD approach.

show abstract

SoloGAN: Multi-domain Multimodal Unpaired Image-to-Image Translation via a Single Generative Adversarial Network

Cited by 8 publications

References 42 publications

Image Translation and Reconstruction Using a Single Dual Mode Lightweight Encoder

Image Translation and Reconstruction Using a Single Dual Mode Lightweight Encoder

Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module

Evolutionary generative design of supercritical airfoils: an automated approach driven by small data

Contact Info

Product

Resources

About