Learning local evidence for shading and reflectance

Abstract

A fundamental, unsolved vision problem is to distinguish image intensity variations caused by surface normal variations from those caused by reflectance changes-ie, to tell shading from paint. A solution to this problem is necessary for machines to interpret images as people do and could have many applications. The labelling allows us to reconstruct bandpassed images containing only those parts of the input image caused by shading effects, and a separate image containing only those parts caused by reflectance changes. The resulting classifications compare well with human psychophysical performance on a test set of images, and show good results for test photographs.This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Abstract A fundamental, unsolved vision problem is to distinguish image intensity v ariations caused by surface normal variations from those caused by re ectance changes ie, to tell shading from paint. A solution to this problem is necessary for machines to interpret images as people do and could have many applications.We take a learning-based approach. We generate a training set of synthetic images containing both shading and re ectance variations. We label the interpretations by indicating which coe cients in a steerable pyramid representation of the image were caused by shading and which b y paint.To analyze local image evidence for shading or re ectance, we study the outputs of two l a y ers of lters, each followed by recti cation. We t a probability density model to the lter outputs using a mixture of factor analyzers. The resulting model indicates the probability, based on local image evidence, that a pyramid coe cient a t a n y orientation and scale was caused by shading or by re ectance variations. We take the lighting direction to be that which generates the most shape-like labelling.The labelling allows us to reconstruct bandpassed images containing only those parts of the input image caused by shading e ects, and a separate image containing only those parts caused by re ectance changes. The resulting classi cations compare well with human psychophysical performance on a test set of images, and show good results for test photographs.This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonpro t educational and research purposes provided that all such whol...

Shape, Illumination, and Reflectance from Shading

“…Over time, "intrinsic images" has become synonymous with the problem that Retinex addressed, that of separating an image into shading and reflectance components [4], [10], [9]. This area has seen seen some recent progress [12], [13], [14], [15], though the performance of Retinex, despite its age, has proven hard to improve upon [4]. The limiting factor in many of these "intrinsic image" algorithms appears to be that they treat "shading" as a kind of image, ignoring the fact that shading is, by construction, the product of some shape and some model of illumination.…”

Correlation-Based Intrinsic Image Extraction from a Single Image

“…The method works well in stylised stimuli where edges are easily defined. With supervised learning, Bell and Freeman [4] reconstructed shading and reflectance from classified steerable filter coefficients.…”

Removing Shadows from Video

“…In recent years, other methods have been developed to remove the effect of illumination from a single image [9]- [12]. These methods are based primarily on distinguishing between the texture of the objects and shadows.…”

“…These methods are based primarily on distinguishing between the texture of the objects and shadows. Bell and Freeman [9] took a learning base approach and generated a training set of images containing Removing Shadows from Video Seyed Mahdi Javadi Brunel, Yongmin Li, and Xiaohui Liu…”