CNN Based Learning Using Reflection and Retinex Models for Intrinsic Image Decomposition

Abstract

Most of the traditional work on intrinsic image decomposition rely on deriving priors about scene characteristics. On the other hand, recent research use deep learning models as in-and-out black box and do not consider the wellestablished, traditional image formation process as the basis of their intrinsic learning process. As a consequence, although current deep learning approaches show superior performance when considering quantitative benchmark results, traditional approaches are still dominant in achieving high qualitative results.In this paper, the aim is to exploit the best of the two worlds. A method is proposed that (1) is empowered by deep learning capabilities, (2) considers a physics-based reflection model to steer the learning process, and (3) exploits the traditional approach to obtain intrinsic images by exploiting reflectance and shading gradient information. The proposed model is fast to compute and allows for the integration of all intrinsic components. To train the new model, an object centered large-scale datasets with intrinsic ground-truth images are created.The evaluation results demonstrate that the new model outperforms existing methods. Visual inspection shows that the image formation loss function augments color reproduction and the use of gradient information produces sharper edges.Datasets, models and higher resolution images are available at https://ivi.fnwi.uva.nl/cv/retinet.

A Survey of Image Synthesis Methods for Visual Machine Learning

“…Early approaches used simple hand‐modelled scenes, populated with either single or few objects, with accurate light transport simulation enabled by photon mapping [BSvdW*13]. Later methods built their scene content using 3D models and scene databases, along with rendering randomization, measured materials and environment maps for global illumination, while employing various flavours of path‐tracing and tone mapping algorithms [RRF*16, LS18, BLG18, BSP*19] (Figure 15a). In the same spirit, Baslamisli et al .…”

Deep intrinsic decomposition trained on surreal scenes yet with realistic light effects

“…This architecture presents one encoder and three decoders with shared connections. A similar approach was followed by Baslamisli et al [6] that introduced another synthetic dataset and two different deep learning architectures named as IntrinsicNet and RetiNet to get shading and reflectance from a single image. IntrinsicNet is an encoder-decoder end-to-end network to predict shading and reflectance from a uniform architecture.…”

“…ShapeNet [5] emerged as a tool to create new larger datasets where synthetic objects are located in multiple different environmental maps. Following this idea, Baslamisli et al [6] used the same approach but using homogeneous reflectance for each object mesh. In both cases the ground-truth is just given by the object area.…”

“…We propose a deep architecture based on the following three criteria: (a) the use of a U-NET-based, Encoder-Decoder, architecture, which has been the most usual [5,6] and natural way to solve this pixel-wise regression problem; (b) increasing efficiency and speed properties by introducing split inception modules decreasing the number of parameters [44]; and (c) introducing physical constraints of the intrinsic decomposition model at the loss function like in [6,34]. We will refer to our architecture as IUI as for Inception U-Net based for Intrinsic image estimation.…”

InverseRenderNet: Learning Single Image Inverse Rendering

“…Ma et al [36] also trained on time-lapse sequences and introduced a new gradient constraint which encourage better explanations for sharp changes caused by shading or reflectance. Baslamisli et al [5] applied a similar gradient constraint while they used supervised training.…”