Reflectance Adaptive Filtering Improves Intrinsic Image Estimation

Abstract

Separating an image into reflectance and shading layers poses a challenge for learning approaches because no large corpus of precise and realistic ground truth decompositions exists. The Intrinsic Images in the Wild (IIW) dataset provides a sparse set of relative human reflectance judgments, which serves as a standard benchmark for intrinsic images. A number of methods use IIW to learn statistical dependencies between the images and their reflectance layer. Although learning plays an important role for high performance, we show that a standard signal processing technique achieves performance on par with current state-of-the-art. We propose a loss function for CNN learning of dense reflectance predictions. Our results show a simple pixel-wise decision, without any context or prior knowledge, is sufficient to provide a strong baseline on IIW. This sets a competitive baseline which only two other approaches surpass. We then develop a joint bilateral filtering method that implements strong prior knowledge about reflectance constancy. This filtering operation can be applied to any intrinsic image algorithm and we improve several previous results achieving a new state-of-the-art on IIW. Our findings suggest that the effect of learning-based approaches may have been over-estimated so far. Explicit prior knowledge is still at least as important to obtain high performance in intrinsic image decompositions.

CGIntrinsics: Better Intrinsic Image Decomposition Through Physically-Based Rendering

“…Comparing direct CNN predictions, our CGI-trained model is significantly better than the best learning-based method [45], and similar to [44], even though [45] was directly trained on IIW. Additionally, running the post-processing from [45] on the results of the CGI-trained model achieves a further performance boost. Table 2 also shows that models trained on SUNCG (i.e., PBRS), Sintel, MIT Intrinsics, or ShapeNet generalize poorly to IIW likely due to the lower quality of training data (SUNCG/PBRS), or the larger domain gap with respect to images of real-world scenes, compared to CGI.…”

InverseRenderNet: Learning Single Image Inverse Rendering

“…Training data WHDR Nestmeyer [38] our shading predictions are limited by the fact that we use an explicit local illumination model (so cannot predict cast shadows). Nevertheless, we test our network on this benchmark directly without fine-tuning.…”

“…Nestmeyer [38] (R) Ours (R) Li [33] (S) Nestmeyer [38] (S) Ours (S) Figure 5: Qualitative results for IIW. Second column to forth column are reflectance predictions from [33], [38] and ours. The last three columns are corresponding shading predictions.…”

Neural Inverse Rendering of an Indoor Scene From a Single Image

“…Intrinsic image decomposition is a sub-problem of inverse rendering, where a single image is decomposed into albedo and shading. Recent methods learn intrinsic image decomposition from labeled synthetic data [17,26,34] and from unlabeled [20] or partially labeled real data [49,19,28,2]. Intrinsic image decomposition methods do not explicitly recover geometry or illumination but rather combine them together as shading.…”

“…Intrinsic image decomposition aims to decompose an image into albedo and shading, which is a subproblem in inverse rendering. Several recent works [2,49,28,19] showed promising results with deep learning. While our goal is to solve the complete inverse rendering problem, we still compare albedo prediction with these latest intrinsic image decomposition methods.…”