Towards Efficient Single Image Dehazing and Desnowing

Tian, Ye; Chen, Sixiang; Liu, Yun; Chen, Erkang; Li, Yuche

doi:10.48550/arxiv.2204.08899

Cited by 4 publications

(6 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, our method introduced a gradient-based desnowing method, which enables the handling of various shapes and trajectories of snow particles based on dual gradients in a single image. Cheng et al [17] accurately removed snow areas from images using on advanced snow masks.Ye et al [18] considered the similarity between dehazing and desnowing problems to solve both problems. MSP-Former [19], LMQFormer [20], and SnowFormer [21] employed transformer-based methods, which enabled to produce accurate desnowing results.…”

Section: Related Workmentioning

confidence: 99%

Dual Gradient Based Snow Attentive Desnowing

et al. 2023

View full text Add to dashboard Cite

In this paper, we propose a novel dual gradient-based desnowing algorithm that can accurately remove snow from a scene by characterizing snow particles. To localize snow in an image, we present a gradient-based snow activation map that can be estimated using snow classification. To recognize various patterns in the shapes and trajectories of snow particles, we introduce a gradient-based snow edge map. Using these two gradients, we estimate an accurate snow attention mask that is subsequently used for snow removal. In addition, we propose a translucency-aware context restoration network to handle various degrees of snow transparency and thus, prevent our method from losing the image context information during desnowing. Experimental results demonstrate that the proposed method considerably outperforms other state-of-the-art desnowing algorithms quantitatively and qualitatively.

show abstract

Section: Related Workmentioning

confidence: 99%

Dual Gradient Based Snow Attentive Desnowing

et al. 2023

View full text Add to dashboard Cite

show abstract

“…The restoration of single images degraded by particles of snow adverse weather is an established ill-posed problem that has attracted a lot of research attention since the era of deep-learning [1], [4], [5], [6], [7], [8], [9], [22], [23], [24], [25], [26], [27], [28], [29]. In [1], Liu et al proposed a multistage context-aware network (dubbed DesnowNet) to deal with the complicated size, density, opaque and translucent characteristics of snow particle removal.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, the emerging unified models designed for multiple noise elements' removal have begun to include desnow tasks in their portfolio. To this end, Ye et al [24] investigated the simultaneous removal of snow and haze with a degradation adaptive network. Likewise, [22] and [26] demonstrate fair success in snow, rain, and haze removal tasks with complex CNN networks.…”

Section: Related Workmentioning

confidence: 99%

Restoring Snow-Degraded Single Images With Wavelet in Vision Transformer

Agbodike,

Chen

2023

IEEE Access

View full text Add to dashboard Cite

Images corrupted by snowy adverse weather can impose performance impediments to critical high-level vision-based applications. Restoring snow-degraded images is vital, but the task is ill-posed and very challenging due to the veiling effect, stochastic distribution, and multi-scale characteristics of snow in a scene. In this regard, many existing image denoising methods are often less successful with respect to snow removal, being that they mostly achieve success with one snow dataset and underperform in others, thus questioning their robustness in tackling real-world complex snowfall scenarios. In this paper, we propose the wavelet in transformer (WiT) network to address the image desnow inverse problem. Our model exploits the joint systemic capabilities of the vision transformer and the renowned discrete wavelet transform to achieve effective restoration of snow-degraded images. In our experiments, we evaluated the performance of our model on the popular SRRS, SNOW100K, and CSD datasets, respectively. The efficacy of our learning-based network is proven by our obtained numeric and qualitative result outcomes indicating significant performance gains compared to image desnow benchmark models and other state-of-the-art methods in the literature. The source code is available at https://github.com/WINS-lab/WiT.

show abstract

“…Figure 2 shows a pipeline with an upstream MoE model to overcome a number of weather effects. For example, Ye et al (2022) propose the DAN-Net method that estimates gated attention maps for inputs and uses them to properly dispatch images to task-specific experts. Similarly, Luo et al (2023) develop a weather-aware router to assign an input image to a relevant expert without a weather-type label at test time.…”

Section: Introductionmentioning

confidence: 99%

Efficient Deweahter Mixture-of-Experts with Uncertainty-Aware Feature-Wise Linear Modulation

Zhang,

Luo,

Liu

et al. 2024

AAAI

View full text Add to dashboard Cite

The Mixture-of-Experts (MoE) approach has demonstrated outstanding scalability in multi-task learning including low-level upstream tasks such as concurrent removal of multiple adverse weather effects. However, the conventional MoE architecture with parallel Feed Forward Network (FFN) experts leads to significant parameter and computational overheads that hinder its efficient deployment. In addition, the naive MoE linear router is suboptimal in assigning task-specific features to multiple experts which limits its further scalability. In this work, we propose an efficient MoE architecture with weight sharing across the experts. Inspired by the idea of linear feature modulation (FM), our architecture implicitly instantiates multiple experts via learnable activation modulations on a single shared expert block. The proposed Feature Modulated Expert (FME) serves as a building block for the novel Mixture-of-Feature-Modulation-Experts (MoFME) architecture, which can scale up the number of experts with low overhead. We further propose an Uncertainty-aware Router (UaR) to assign task-specific features to different FM modules with well-calibrated weights. This enables MoFME to effectively learn diverse expert functions for multiple tasks. The conducted experiments on the multi-deweather task show that our MoFME outperforms the state-of-the-art in the image restoration quality by 0.1-0.2 dB while saving more than 74% of parameters and 20% inference time over the conventional MoE counterpart. Experiments on the downstream segmentation and classification tasks further demonstrate the generalizability of MoFME to real open-world applications.

show abstract

Towards Efficient Single Image Dehazing and Desnowing

Abstract: Figure 1: Realistic winter images (top) from the RWSD dataset created by this work and corresponding restoration results (bottom) using the proposed Degradation-Adaptive Neural Network.

Cited by 4 publications

References 34 publications

Dual Gradient Based Snow Attentive Desnowing

Dual Gradient Based Snow Attentive Desnowing

Restoring Snow-Degraded Single Images With Wavelet in Vision Transformer

Efficient Deweahter Mixture-of-Experts with Uncertainty-Aware Feature-Wise Linear Modulation

Contact Info

Product

Resources

About