A Survey of Deep Learning-Based Image Restoration Methods for Enhancing Situational Awareness at Disaster Sites: The Cases of Rain, Snow and Haze

Abstract

This survey article is concerned with the emergence of vision augmentation AI tools for enhancing the situational awareness of first responders (FRs) in rescue operations. More specifically, the article surveys three families of image restoration methods serving the purpose of vision augmentation under adverse weather conditions. These image restoration methods are: (a) deraining; (b) desnowing; (c) dehazing ones. The contribution of this article is a survey of the recent literature on these three problem families, focusing on the utilization of deep learning (DL) models and meeting the requirements of their application in rescue operations. A faceted taxonomy is introduced in past and recent literature including various DL architectures, loss functions and datasets. Although there are multiple surveys on recovering images degraded by natural phenomena, the literature lacks a comprehensive survey focused explicitly on assisting FRs. This paper aims to fill this gap by presenting existing methods in the literature, assessing their suitability for FR applications, and providing insights for future research directions.

Weather removal with a lightweight quaternion Chebyshev neural network

“…Recent progress in adverse weather removal is related to introducing methods based on deep-learning [1] . In this work, we focus on methods able to handle various degradations.…”

“…Visual perception plays a vital role in generating the intelligent actions of autonomous vehicles. Computer vision systems are critical in various real-world applications, such as outdoor surveillance, highway monitoring, and autonomous vehicles [1][2][3][4][41][42][43]. However, adverse weather conditions such as haze, rain, or adherent raindrops significantly affect performance of these systems, especially of state-of-the-art high-level computer vision techniques.…”

“…Recent breakthroughs in neural network compression techniques have shown great potential in reducing the number of trainable parameters and training from scratch with limited data. However, while techniques such as harmonic convolutional networks, vision transformers (ViT), and quaternion neural networks have shown promise, they have found little use in real-world applications [1], [2].…”

Data-driven polarimetric imaging: a review

“…Clear vision in scattering media is critical for various applications such as industrial and civil fields 130 , traffic surveillance systems 131 , automatic drives 132 , remote sensing 133 , rescue operations 134 , seabed mapping 135 , monitoring of marine species migration and coral reefs, and scene analysis 136 . However, when capturing images in a scattered environment, the visibility of objects is typically sharply degraded, which is caused by the scattering of suspended particles such as clouds, water, haze, smoke, smog, fog, and mist in the air, soil particles, floating excrement of marine animals, algae, and mineral salt in underwater scenes.…”

Single‐image snow removal algorithm based on generative adversarial networks

“…In general, the foggy occlusion effect produced by heavy snowfall can reduce visibility in a scene. In blizzard conditions, dense snow marks can severely degrade the reliability of machine vision systems for various tasks, such as target detection [1] and semantic segmentation [2]. According to the work of Chen et al.…”