Very Deep Learning-Based Illumination Estimation Approach With Cascading Residual Network Architecture (CRNA)

Abstract

For the imaging signal processing (ISP) pipeline of digital image devices, it is of high significance to remove undesirable illuminant effects and obtain color invariance, commonly known as 'computational color constancy'. Achieving the computational color constancy requires going through two phases: the illumination estimation, which will be the primary focus of this work, and the human visual perception-based chromatic adaptation. At the first phase, illumination estimation is to predict RGB triplets, the numeric representations of incident illuminant colors, by calculating the values of image pixels. How much the network can increase its estimation accuracy is a key to realizing computational color constancy. With recent advances in deep learning (DL), a lot of deep learning-based approaches have been suggested, bringing higher accuracy to computer vision applications, but there are still quite a few obstacles to overcome such as instability of learning. In an attempt to address this ill-posed problem in the illumination estimation space, this article presents a novel deep learning-based approach, the Cascading Residual Network Architecture (CRNA), which incorporates the ResNet and cascading mechanism into the deep convolutional neural network (DCNN). The cascading mechanism helps the proposed network to restrain from suddenly varying in size, serves to mitigate learning instability, and thereby reduces the quality degradation. This is attributed to the ability of the cascading mechanism to fine-tune the pre-trained DCNN. Considerable amounts of datasets and comparative experiments highlight that the proposed approach delivers more stable and robust results and imply the potential for generalization of the proposed approach across deep learning applications.INDEX TERMS image signal processing pipeline, computational color constancy, learning instability, cascading mechanism, illumination estimation.

Framework for Illumination Estimation and Segmentation in Multi-Illuminant Scenes

“…The best performing multi-illuminant model is highlighted in yellow. Data for single-illuminant models was obtained from [15] and [16]. (Lower is better.…”

“…This approach was successful because it allowed the model to reason about the patches of the image that carry more information about the color of the illumination. In [16], the authors propose a very deep model for illuminant estimation (CRNA) that uses cascading residual connections and ResNet architecture to stabilize learning and improve performance. Similarly, in [17], the authors propose a deep network which iteratively estimates the illumination, which is also used to stabilize training and improve performance.…”