Normal Integration: A Survey

Abstract

The need for efficient normal integration methods is driven by several computer vision tasks such as shape-from-shading, photometric stereo, deflectometry, etc. In the first part of this survey, we select the most important properties that one may expect from a normal integration method, based on a thorough study of two pioneering works by Horn and Brooks [28] and by Frankot and Chellappa [19]. Apart from accuracy, an integration method should at least be fast and robust to a noisy normal field. In addition, it should be able to handle several types of boundary condition, including the case of a free boundary, and a reconstruction domain of any shape i.e., which is not necessarily rectangular. It is also much appreciated that a minimum number of parameters have to be tuned, or even no parameter at all. Finally, it should preserve the depth discontinuities. In the second part of this survey, we review most of the existing methods in view of this analysis, and conclude that none of them satisfies all of the required properties. This work is complemented by a companion paper entitled Variational Methods for Normal Integration, in which we focus on the problem of normal integration in the presence of depth discontinuities, a problem which occurs as soon as there are occlusions.

A Variational Approach to Shape-from-Shading Under Natural Illumination

“…However, since integrability is not strictly enforced but only used as regularization, there is no guarantee for θ to be integrable. Therefore, Horn and Brooks recast the integration task as another variational problem (see [14] for an overview of this problem):…”

Photometric Depth Super-Resolution

“…Denoting by f > 0 the focal length of the color camera, and by p : Ω HR → R 2 the field of pixel coordinates with respect to its principal point (blue reference coordinates system in Figure 2), the surface normal is defined as the following Ω HR → S 2 ⊂ R 3 field of unit-length vectors (see e.g., [3]):…”

Variational Uncalibrated Photometric Stereo Under General Lighting

“…Such a differential approach to photometric stereo can be coupled with variational methods in order to iteratively refine depth, reflectance and lighting in a robust manner [42]. In addition to the theoretical interest of enforcing integrability in order to limit ambiguities, differential approaches to photometric stereo have the advantages of easing combination with other 3D-reconstruction methods [17,40], and of bypassing the problem of integrating the estimated normal field, which is by itself a non-trivial problem [41]. Besides, any error in the estimated normal field might propagate during integration, and thus robustness to specularities or shadows must be enforced during normal estimation, see again [48] for some discussion.…”

“…2. We then convert the orthographic depth z o to a perspective depth z p via normal integration [41].…”

“…The overall pipeline in Phase 2 is summarized as follows: 3. (∇ z p → z p ): Perform integration [41] to obtain z p . Return z p = exp z p as the initialized (perspective) depth.…”