Confidence Measure Guided Single Image De-Raining

Yasarla, Rajeev; Patel, Vishal M.

doi:10.1109/tip.2020.2973802

Cited by 62 publications

(19 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The deraining architecture is proposed based on the effectiveness of rain-streak location information in a multiscale fashion rather than about the density information. A new approach based on the distortion level of each image patch at different locations using a confidence measure guided training technique is proposed in [41]. An encoder-decoder based deraining architecture is proposed in [42] where the decoder layers are conditioned on the embedding learned by a separate branch towards encoder side.…”

Section: B Data-driven Methodsmentioning

confidence: 99%

Recursive Multi-Scale Image Deraining With Sub-Pixel Convolution Based Feature Fusion and Context Aggregation

et al. 2020

View full text Add to dashboard Cite

Along with several other low-vision based computer vision problems, single image deraining is also taken as a challenging one due to its ill-posedness. Several algorithms based on convolutional neural networks are devised that are either too simple to provide acceptable deraining results due to under-deraining or have a complex architectures that may result into over-deraining. In this paper we propose a deraining algorithm that is capable of boosting the reconstruction/deraining quality without the problem of over or under-deraining. Along with the originally proposed network, two of its' lightweight versions with reduced computational costs are also devised. Basically, we propose a recursively trained architecture that has two major components: a front-end module and a refinement module. The front-end module is based on dense fusion of lower label features followed by sub-pixel convolutions (pixel shuffling based convolutions). To refine and generate the enhanced deraining results further, we cascade a refinement module to the frontend module using multi-scale Context Aggregation Network (CAN) which includes feature fusion and pixel shuffling based convolutions. We present the deraining results in terms of Structural Similarity Index (SSIM) and Peak Signal to Noise Ratio (PSNR) on several benchmarks and compare with current state-of-theart algorithms. With comprehensive experiments on both real-world and synthetic datasets and extensive ablation study, we demonstrate that our approach produces better results compared to existing methods. INDEX TERMS Single image deraining, multi-scale context aggregation, sub-pixel convolution, feature fusion.

show abstract

Section: B Data-driven Methodsmentioning

confidence: 99%

Recursive Multi-Scale Image Deraining With Sub-Pixel Convolution Based Feature Fusion and Context Aggregation

et al. 2020

View full text Add to dashboard Cite

show abstract

“…In contrast, we use the Effective Nearest Neighbour (ENN) based sharpness measure introduced in [20] to find the sharpest input frame and use it as the reference to register the input frames. Further we use a confidence guided loss [21] to train a U-Net [22] based network to restore (i.e. deblur) the registered frames to obtain a sharp output.…”

Section: Distorted Imagementioning

confidence: 99%

“…Inspired by modeling the data dependent aleatoric uncertainty [29,30], [21] used it as a confidence guided loss for deraining. In our network we also use a modified version of the confidence map loss defined by…”

Section: Lossesmentioning

confidence: 99%

Confidence Guided Network For Atmospheric Turbulence Mitigation

Gopalakrishnan¹,

Patel

2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

Atmospheric turbulence can adversely affect the quality of images or videos captured by long range imaging systems. Turbulence causes both geometric and blur distortions in images which in turn results in poor performance of the subsequent computer vision algorithms like recognition and detection. Existing methods for atmospheric turbulence mitigation use registration and deconvolution schemes to remove degradations. In this paper, we present a deep learning-based solution in which Effective Nearest Neighbors (ENN) based method is used for registration and an uncertainty-based network is used for restoration. We perform qualitative and quantitative comparisons using synthetic and real-world datasets to show the significance of our work.

show abstract

“…In this section, we begin with a description of the proposed network architecture. We largely follow the encoder-decoder network, which has been widely adopted for existing single image de-raining [6], [9], [14], [60]. We describe a detailed description of our network architecture in Table I.…”

Section: B Network Architecturementioning

confidence: 99%

“…Hu et al [46], [47] proposed a depth-guided attention mechanism to remove rain and fog. Yasarla et al [60] proposed an image quality-based method that learns the quality or distortion level of each patch in the rain image. MPRNet [19] proposed a multi-stage architecture that progressively learns restoration functions for the degraded inputs.…”

Section: Related Work a Single Image De-rainingmentioning

confidence: 99%

Single Image Deraining Using Time-Lapse Data

Jaehoon

Kim

Min

et al. 2020

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

This paper addresses the problem of single image de-raining, that is, the task of recovering clean and rain-free background scenes from a single image obscured by a rainy artifact. Although recent advances adopt real-world time-lapse data to overcome the need for paired rain-clean images, they are limited to fully exploit the time-lapse data. The main cause is that, in terms of network architectures, they could not capture long-term rain streak information in the time-lapse data during training owing to the lack of memory components. To address this problem, we propose a novel network architecture based on a memory network that explicitly helps to capture long-term rain streak information in the time-lapse data. Our network comprises the encoder-decoder networks and a memory network. The features extracted from the encoder are read and updated in the memory network that contains several memory items to store rain streak-aware feature representations. With the read/update operation, the memory network retrieves relevant memory items in terms of the queries, enabling the memory items to represent the various rain streaks included in the time-lapse data. To boost the discriminative power of memory features, we also present a novel background selective whitening (BSW) loss for capturing only rain streak information in the memory network by erasing the background information. Experimental results on standard benchmarks demonstrate the effectiveness and superiority of our approach.

show abstract

Confidence Measure Guided Single Image De-Raining

Cited by 62 publications

References 41 publications

Recursive Multi-Scale Image Deraining With Sub-Pixel Convolution Based Feature Fusion and Context Aggregation

Recursive Multi-Scale Image Deraining With Sub-Pixel Convolution Based Feature Fusion and Context Aggregation

Confidence Guided Network For Atmospheric Turbulence Mitigation

Single Image Deraining Using Time-Lapse Data

Contact Info

Product

Resources

About