Exploiting Global and Local Attentions for Heavy Rain Removal on Single Images

Abstract

Heavy rain removal from a single image is the task of simultaneously eliminating rain streaks and fog, which can dramatically degrade the quality of captured images. Most existing rain removal methods do not generalize well for the heavy rain case. In this work, we propose a novel network architecture consisting of three sub-networks to remove heavy rain from a single image without estimating rain streaks and fog separately. The first sub-net, a U-net-based architecture that incorporates our Spatial Channel Attention (SCA) blocks, extracts global features that provide sufficient contextual information needed to remove atmospheric distortions caused by rain and fog. The second sub-net learns the additive residues information, which is useful in removing rain streak artifacts via our proposed Residual Inception Modules (RIM). The third sub-net, the multiplicative sub-net, adopts our Channel-attentive Inception Modules (CIM) and learns the essential brighter local features which are not effectively extracted in the SCA and additive sub-nets by modulating the local pixel intensities in the derained images. Our three clean image results are then combined via an attentive blending block to generate the final clean image. Our method with SCA, RIM, and CIM significantly outperforms the previous state-of-the-art single-image deraining methods on the synthetic datasets, shows considerably cleaner and sharper derained estimates on the real image datasets. We present extensive experiments and ablation studies supporting each of our method's contributions on both synthetic and real image datasets.INDEX TERMS Heavy rain removal, Single image deraining, Image restoration, Deep learning.

Rain Streaks Removal in Images using Extended Generative Adversarial-based Deraining Framework

“…where, q u refers to the input, the activation function is indicated as squashing function and is represented as q w , in which the long vector is shrunken into the required length based on  and the short vector is shrunken to the length zero. The CapsuleNet acquires the input as,   p p pq pq q v Wei s u (10) Where, the weight factor is notated as Where, the probabilities of two coupled capsules are notated as pq…”

“…However, the available rain streaks can blur the scene and degrade the performance of such techniques [8]. The major problem in deraining approaches is exploiting the necessary attributes of rain streaks and the pure image [9][10][11][12]. The model-based methods define the removal of rain streaks as an optimization issue [13].…”

Spatial and Spectral-Channel Attention Network for Denoising on Hyperspectral Remote Sensing Image

“…To approximate the NSS of HSI, the spatial attention (SA) block [19] is employed to model the connection of different spatial regions (or pixels) by adaptively learning spatial weights. To model the GCS of HSIs, the spectral-channel attention (SCA) block is used to model the global correlation along spectral dimension also by adaptively learning the spectral weights [20]. Particularly, we find that exchanging the dimensions of channels and bands before global average pooling can better promote the effectiveness of attention on a channel.…”