A Simple Model for Intrinsic Image Decomposition with Depth Cues

Chen, Qifeng; Koltun, Vladlen

doi:10.1109/iccv.2013.37

Cited by 170 publications

(148 citation statements)

References 36 publications

Supporting

Mentioning

148

Contrasting

Order By: Relevance

“…The quality of the decompositions can be improved by imposing priors on non-local reflectance [Shen et al 2008;Gehler et al 2011;Zhao et al 2012;Bell et al 2014], on gradient distributions [Li and Brown 2014;Shen and Yeo 2011], on the number of reflectance colors [Shen and Yeo 2011], or on the underlying scene geometry and illumination [Barron and Malik 2012]. Alternatively, the conditioning of the problem can be improved by leveraging multiple images captured under varying illumination [Weiss 2001;Matsushita et al 2004;Laffont et al 2012] or view directions [Laffont et al 2013], or by making use of depth information captured with RGBD cameras [Lee et al 2012;Barron and Malik 2013;Chen and Koltun 2013].…”

Section: Related Workmentioning

confidence: 99%

“…Nonetheless, significant progress has been made on this problem recently, and usable decompositions of real-world scenes have been obtained by incorporating additional data such as depth maps [Lee et al 2012;Barron and Malik 2013;Chen and Koltun 2013], multiple images of the same scene [Weiss 2001;Laffont et al 2012], or sophisticated priors such as reflectance sparsity [Gehler et al 2011] and non-local texture cues [Shen et al 2008;Garces et al 2012;Laffont et al 2012;Zhao et al 2012]. Incorporating these additional terms typically result in sophisticated algorithms which take minutes to solve for a single image.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Interactive intrinsic video editing

et al. 2014

View full text Add to dashboard Cite

. Now, editing the textures in the reflectance image does not affect the illumination (c): changes to the brick walls, the roof tiles, and the pathway leading up to the building all maintain the complex illumination of the light through the trees. We encourage readers to zoom into this figure to see the details in our results, and refer them to the accompanying video to see the temporally consistent nature of our decomposition. AbstractSeparating a photograph into its reflectance and illumination intrinsic images is a fundamentally ambiguous problem, and state-of-the-art algorithms combine sophisticated reflectance and illumination priors with user annotations to create plausible results. However, these algorithms cannot be easily extended to videos for two reasons: first, naïvely applying algorithms designed for single images to videos produce results that are temporally incoherent; second, effectively specifying user annotations for a video requires interactive feedback, and current approaches are orders of magnitudes too slow to support this. We introduce a fast and temporally consistent algorithm to decompose video sequences into their reflectance and illumination components. Our algorithm uses a hybrid 2-p formulation that separates image gradients into smooth illumination and sparse reflectance gradients using look-up tables. We use a multi-scale parallelized solver to reconstruct the reflectance and illumination from these gradients while enforcing spatial and temporal reflectance constraints and user annotations. We demonstrate that our algorithm automatically produces reasonable results, that can be interactively refined by users, at rates that are two orders of magnitude faster than existing tools, to produce high-quality decompositions for challenging real-world video sequences. We also show how these decompositions can be used for a number of video editing applications including recoloring, retexturing, illumination editing, and lighting-aware compositing.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Interactive intrinsic video editing

et al. 2014

View full text Add to dashboard Cite

show abstract

“…In the supplementary material we show additional scenes and also provide results computed with the algorithm of Chen et al [5]. This algorithm reports state of the art performance for RGB-D intrinsic image decomposition only utilizing color and depth information.…”

Section: Discussionmentioning

confidence: 99%

“…As this is an ill-posed problem strong priors and involved optimization algorithms are required. Subsequently, other approaches tried to simplify the problem by including scene geometry from RGB-D cameras [1,5]. Recently, Chen et al proposed in addition to model shadows explicitly [18].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Towards Illumination-Invariant 3D Reconstruction Using ToF RGB-D Cameras

Kerl¹,

Souiai²,

Sturm³

et al. 2014

2014 2nd International Conference on 3D Vision

View full text Add to dashboard Cite

Creating textured 3D scans of indoor environments has experienced a large boost with the advent of cheap commodity depth sensors. However, the quality of the acquired 3D models is often impaired by color seams in the reconstruction due to varying illumination (e.g., shadows or highlights) and object surfaces whose brightness and color vary with the viewpoint of the camera. In this paper, we propose a direct and simple method to estimate the pure albedo of the texture, which allows us to remove illumination effects from IR and color images. Our approach first computes the illumination-independent albedo in the IR domain, which we subsequently transfer to the color albedo. As shadows and highlights lead to over-and underexposed image regions with little or no color information, we apply an advanced optimization scheme to infer color information in the color albedo from neighboring image regions. We demonstrate the applicability of our approach to various real-world scenes.

show abstract

Interactive light source position estimation for augmented reality with an RGB‐D camera

Boom

Orts-Escolano

Ning

et al. 2015

Computer Animation & Virtual

View full text Add to dashboard Cite

The first hybrid CPU‐GPU based method for estimating a point light source position in a scene recorded by an RGB‐D camera is presented. The image and depth information from the Kinect is enough to estimate a light position in a scene, which allows for the rendering of synthetic objects into a scene that appears realistic enough for augmented reality purposes. This method does not require a light probe or other physical device. To make this method suitable for augmented reality, we developed a hybrid implementation that performs light estimation in under 1second. This is sufficient for most augmented reality scenarios because both the position of the light source and the position of the Kinect are typically fixed. The method is able to estimate the angle of the light source with an average error of 20°. By rendering synthetic objects into the recorded scene, we illustrate that this accuracy is good enough for the rendered objects to look realistic. Copyright © 2015 John Wiley & Sons, Ltd.

show abstract

A Simple Model for Intrinsic Image Decomposition with Depth Cues

Cited by 170 publications

References 36 publications

Interactive intrinsic video editing

Interactive intrinsic video editing

Towards Illumination-Invariant 3D Reconstruction Using ToF RGB-D Cameras

Interactive light source position estimation for augmented reality with an RGB‐D camera

Contact Info

Product

Resources

About