Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation

Ye, Yuntong; Chang, Yi; Zhou, Hanyu; Yan, Luxin

doi:10.1109/cvpr46437.2021.00209

Cited by 88 publications

(61 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The good part is that the generator can automatically generate diverse and non-repetitive training pairs so that efficiency is ensured. Similar rain generation is proposed by Ye et al [158] using disentangled image translation to close the loop. Furthermore, Z. Yue et al [157] surpassed image frames and achieved semisupervised video de-raining with a dynamic rain generator.…”

Section: Rainmentioning

confidence: 93%

“…Histogram of oriented gradient (HOG) and autocorrelation loss are used to facilitate the orientation consistency and repress repetitive rain streaks. They trained the network all the way from drizzle to downpour rain Fusion [110] LiDAR [152] LiDAR [76] LiDAR [153] Others [154] LiDAR [155] LiDAR [156] Camera [157] Camera [158] Camera [159] Camera [160] Camera [161] Camera [162] Camera [163] Camera [164] LiDAR [165] LiDAR [166] LiDAR [128] LiDAR [29] Fusion [129] LiDAR [167] Fusion [168] LiDAR [169] LiDAR [170] Fusion [171] LiDAR [172] Camera [173] Camera [174] Camera [175] Camera [176] Camera [177] Camera [178] Camera [179] Camera [180] Camera [181] Camera [182] Camera [183] Camera [184] Camera [185] Camera [186] Fusion [187] Fusion [188] LiDAR [189] Camera [190] Camera…”

Section: Rainmentioning

confidence: 99%

See 1 more Smart Citation

Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey

Zhang¹,

Carballo²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Automated Driving Systems (ADS) open up a new domain for the automotive industry and offer new possibilities for future transportation with higher efficiency and comfortable experiences. However, autonomous driving under adverse weather conditions has been the problem that keeps autonomous vehicles (AVs) from going to level 4 or higher autonomy for a long time. This paper assesses the influences and challenges that weather brings to ADS sensors in an analytic and statistical way, and surveys the solutions against inclement weather conditions. Stateof-the-art techniques on perception enhancement with regard to each kind of weather are thoroughly reported. External auxiliary solutions like V2X technology, weather conditions coverage in currently available datasets, simulators, and experimental facilities with weather chambers are distinctly sorted out. By pointing out all kinds of major weather problems the autonomous driving field is currently facing, and reviewing both hardware and computer science solutions in recent years, this survey contributes a holistic overview on the obstacles and directions of ADS development in terms of adverse weather driving conditions.

show abstract

Section: Rainmentioning

confidence: 93%

Section: Rainmentioning

confidence: 99%

Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey

Zhang¹,

Carballo²,

Yang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Most disentanglement frameworks attempt to learn representations which capture different factors of variation in the latent space. Recently, Ye et al [51] decomposed the rainy image into the rain-free background and the rain layer in disentangle image translation framework. Inspired by [34], we flexibly embed the contrastive learning into the disentangle translation network to enable the end-to-end training, which could be beneficial to image disentanglement.…”

Section: Disentanglement In Gansmentioning

confidence: 99%

Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning

Chen¹,

Fan²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present an effective unpaired learning based image dehazing network from an unpaired set of clear and hazy images. This paper provides a new perspective to treat image dehazing as a two-class separated factor disentanglement task, i.e, the task-relevant factor of clear image reconstruction and the task-irrelevant factor of haze-relevant distribution. To achieve the disentanglement of these two-class factors in deep feature space, contrastive learning is introduced into a CycleGAN framework to learn disentangled representations by guiding the generated images to be associated with latent factors. With such formulation, the proposed contrastive disentangled dehazing method (CDD-GAN) first develops negative generators to cooperate with the encoder network to update alternately, so as to produce a queue of challenging negative adversaries. Then these negative adversaries are trained end-to-end together with the backbone representation network to enhance the discriminative information and promote factor disentanglement performance by maximizing the adversarial contrastive loss. During the training, we further show that hard negative examples can suppress the task-irrelevant factors and unpaired clear exemples can enhance the task-relevant factors, in order to better facilitate haze removal and help image restoration. Extensive experiments on both synthetic and real-world datasets demonstrate that our method performs favorably against existing stateof-the-art unpaired dehazing approaches.

show abstract

“…At present, as shown in Fig. 1 (a), most rain removal methods [1], [20], [47] are based on a supervised learning structure. They first design a rain streak extraction network to extract the rain streak feature from the rainy image.…”

Section: Introductionmentioning

confidence: 99%

An Unsupervised Attentive-Adversarial Learning Framework for Single Image Deraining

Liu¹,

Jiang²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Single image deraining has been an important topic in low-level computer vision tasks. The atmospheric veiling effect (which is generated by rain accumulation, similar to fog) usually appears with the rain. Most deep learning-based single image deraining methods mainly focus on rain streak removal by disregarding this effect, which leads to low-quality deraining performance. In addition, these methods are trained only on synthetic data, hence they do not take into account real-world rainy images. To address the above issues, we propose a novel unsupervised attentive-adversarial learning framework (UALF) for single image deraining that trains on both synthetic and real rainy images while simultaneously capturing both rain streaks and rain accumulation features. UALF consists of a Rain-fog2Clean (R2C) transformation block and a Clean2Rainfog (C2R) transformation block. In R2C, to better characterize the rain-fog fusion feature and to achieve high-quality deraining performance, we employ an attention rain-fog feature extraction network (ARFE) to exploit the self-similarity of global and local rain-fog information by learning the spatial feature correlations. Moreover, to improve the transformation ability of C2R, we design a rain-fog feature decoupling and reorganization network (RFDR) by embedding a rainy image degradation model and a mixed discriminator to preserve richer texture details. Extensive experiments on benchmark rain-fog and rain datasets show that UALF outperforms state-of-the-art deraining methods. We also conduct defogging performance evaluation experiments to further demonstrate the effectiveness of UALF.

show abstract

Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation

Cited by 88 publications

References 39 publications

Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey

Perception and Sensing for Autonomous Vehicles Under Adverse Weather Conditions: A Survey

Unpaired Deep Image Dehazing Using Contrastive Disentanglement Learning

An Unsupervised Attentive-Adversarial Learning Framework for Single Image Deraining

Contact Info

Product

Resources

About