Dual Contrastive Learning for Unsupervised Image-to-Image Translation

Han, Junlin; Shoeiby, Mehrdad; Petersson, Lars; Armin, Mohammad Ali

doi:10.1109/cvprw53098.2021.00084

Cited by 169 publications

(157 citation statements)

References 25 publications

Supporting

Mentioning

157

Contrasting

Order By: Relevance

“…Domain adaptation is one type of transfer learning and has been well explored in a centralized setting where both the source domain and target domain data are available for performing knowledge transfer. In this regard, a reconstruction-based method with an encoder-decoder architecture aims to learn a discriminative mapping of target samples to the source feature space thus improving generalization performance Tzeng et al (2017); Ghifary et al (2016); Han et al (2021). However, the generative approach is generally resource-consuming relaying on computational capability.…”

Section: Related Workmentioning

confidence: 99%

Feature Distribution Matching for Federated Domain Generalization

Sun¹,

Chong²,

Ochiai³

2022

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) facilitates distributed model learning to protect users' privacy. In the absence of labels for a new user's data, the knowledge transfer in FL allows a learned global model to adapt to the new samples quickly. The multi-source domain adaptation in FL aims to improve the model's generality in a target domain by learning domain-invariant features from different clients. In this paper, we propose Federated Knowledge Alignment (FedKA) that aligns features from different clients and those of the target task. We identify two types of negative transfer arising in multi-source domain adaptation of FL and demonstrate how FedKA can alleviate such negative transfers with the help of a global features disentangler enhanced by embedding matching. To further facilitate representation learning of the target task, we devise a federated voting mechanism to provide labels for samples from the target domain via a consensus from querying local models and fine-tune the global model with these labeled samples. Extensive experiments, including an ablation study, on an image classification task of Digit-Five and a text sentiment classification task of Amazon Review, show that FedKA could be augmented to existing FL algorithms to improve the generality of the learned model for tackling a new task.Preprint. Under review.

show abstract

Section: Related Workmentioning

confidence: 99%

Feature Distribution Matching for Federated Domain Generalization

Sun¹,

Chong²,

Ochiai³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…It aims to maximizes the mutual information by pulling the anchor close to positive samples while pushing it away from negative samples in the representation space. Recent studies have employed contrastive learning into low-level vision tasks and obtained improved performance, such as image dehazing [48], image deraining [7], image super-resolution [44] and image-to-image translation [35,14]. The most critical design in contrastive learning is how to select the negatives.…”

Section: Contrastive Learningmentioning

confidence: 99%

Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning

Chen¹,

Fan²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present an effective unpaired learning based image dehazing network from an unpaired set of clear and hazy images. This paper provides a new perspective to treat image dehazing as a two-class separated factor disentanglement task, i.e, the task-relevant factor of clear image reconstruction and the task-irrelevant factor of haze-relevant distribution. To achieve the disentanglement of these two-class factors in deep feature space, contrastive learning is introduced into a CycleGAN framework to learn disentangled representations by guiding the generated images to be associated with latent factors. With such formulation, the proposed contrastive disentangled dehazing method (CDD-GAN) first develops negative generators to cooperate with the encoder network to update alternately, so as to produce a queue of challenging negative adversaries. Then these negative adversaries are trained end-to-end together with the backbone representation network to enhance the discriminative information and promote factor disentanglement performance by maximizing the adversarial contrastive loss. During the training, we further show that hard negative examples can suppress the task-irrelevant factors and unpaired clear exemples can enhance the task-relevant factors, in order to better facilitate haze removal and help image restoration. Extensive experiments on both synthetic and real-world datasets demonstrate that our method performs favorably against existing stateof-the-art unpaired dehazing approaches.

show abstract

“…We focus on unsupervised image-to-image translation models to fully exploit HICRD. We compare CWR to several state-of-the-art baselines from different views, including image-to-image translation approaches (CUT [43], CycleGAN [44] and DCLGAN [49]), conventional underwater image enhancement methods (Histogramprior [50], Retinex [51] and Fusion [52]), conventional underwater image restoration methods (UDCP [9], DCP [8], IBLA [13], and Haze-line [15]), and learning-based restoration method (UWCNN [20]). We use the pre-trained UWCNN model with water type-3, which is close to our dataset.…”

Section: A Baselinesmentioning

confidence: 99%

“…For image-to-image translation approaches, CUT [43] and DCLGAN [49] aim to maximize the mutual information between corresponding patches of the input and the output. DCLGAN [49] employs a dual learning setting and assigns different encoders to different domains, to gain a better performance. We firstly employ CUT and DCLGAN to enable the underwater image restoration task.…”

Section: A Baselinesmentioning

confidence: 99%

Single Underwater Image Restoration by Contrastive Learning

Han

Shoeiby

Malthus

et al. 2021

2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS

Self Cite

View full text Add to dashboard Cite

Underwater image restoration is of significant importance in unveiling the underwater world. Numerous techniques and algorithms have been developed in the past decades. However, due to fundamental difficulties associated with imaging/sensing, lighting, and refractive geometric distortions, in capturing clear underwater images, no comprehensive evaluations have been conducted of underwater image restoration. To address this gap, we have constructed a large-scale real underwater image dataset, dubbed 'HICRD' (Heron Island Coral Reef Dataset), for the purpose of benchmarking existing methods and supporting the development of new deep-learning based methods. We employ accurate water parameter (diffuse attenuation coefficient) in generating reference images. There are 2000 reference restored images and 6003 original underwater images in the unpaired training set. Further, we present a novel method for underwater image restoration based on unsupervised image-to-image translation framework. Our proposed method leveraged contrastive learning and generative adversarial networks to maximize the mutual information between raw and restored images. Extensive experiments with comparisons to recent approaches further demonstrate the superiority of our proposed method. Our code and dataset are publicly available at GitHub.

show abstract

Dual Contrastive Learning for Unsupervised Image-to-Image Translation

Cited by 169 publications

References 25 publications

Feature Distribution Matching for Federated Domain Generalization

Feature Distribution Matching for Federated Domain Generalization

Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning

Single Underwater Image Restoration by Contrastive Learning

Contact Info

Product

Resources

About