A linear formulation of shape from specular flow

Canas, Guillermo D.; Vasilyev, Yuriy; Adato, Yair; Zickler, Todd; Gortler, Steven J.; Ben-Shahar, Ohad

doi:10.1109/iccv.2009.5459164

Cited by 28 publications

(41 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most recent contributions to such problems have exploited the specular flow -the vector field that is induced on the image plane as a result of a relative motion between the camera, object, or environment -to facilitate diverse tasks such as shape inference [1,9,17,20], 3D pose estimation [10] and detection of rigid objects [23]. However, while all these studies clearly demonstrate that specular flows can be exploited successfully (and theoretically rigorously) in various computational ways, they do not address the problem of how specular flows can be estimated from image data.…”

Section: Introductionmentioning

confidence: 99%

Toward robust estimation of specular flow

Adato¹,

Zickler²,

Ben-Shahar³

2010

Procedings of the British Machine Vision Conference 2010

Self Cite

View full text Add to dashboard Cite

Specular flow is an important class of optical flow whose utility in visual tasks has gained much interest in contemporary vision research. Unfortunately, however, reliably estimation of specular flow from image sequences is an open question that was never addressed formally before. In this paper we first argue that existing optical flow algorithms are incapable of reliable specular flow estimation due to their typical regularization criteria that conflict the unique and singular structure of specular flows. We show these discrepancies both qualitatively and using quantitative evaluation based on a firstof-its-kind benchmark dataset with ground truth specular flow data. We then suggest to generalize the popular optical flow variational framework using spatial weighting of different regularizers, and we propose new regularization terms that correspond better to the expected singularities of specular flows. Finally, we show how these contributions significantly improves specular flow estimation.

show abstract

Section: Introductionmentioning

confidence: 99%

Toward robust estimation of specular flow

Adato¹,

Zickler²,

Ben-Shahar³

2010

Procedings of the British Machine Vision Conference 2010

Self Cite

View full text Add to dashboard Cite

show abstract

“…In most cases, however, it is not known how to solve SFSF equation or how to obtain the initial conditions. Some progress related to the former problem was presented by Canas et al [9], who proposed to represent the unknown shape via the field of its reflection vectors, r ∈ S 2 Figure 1: Evaluation of the basic algorithm. (a) The setup: a specular surface is observed orthographically in a distant unknown illumination environment.…”

Section: Previous Workmentioning

confidence: 99%

“…Regardless of which SFSF one attempts to solve, one question remains -how can one measure or estimate the specular flow reliably and accurately enough from image sequences. Since specular flows are, by definition, optical flows [11], it was first proposed to employ optical flow algorithms for this task (e.g., [1,9]). Unfortunately, however, this is a poor choice.…”

Section: Previous Workmentioning

confidence: 99%

“…In order to formulate the constraints between the unknown surface and flow, one should choose an imaging model. We follow the model proposed in the SFSF literature [1,2,9,22], to enjoy the advantages of the linear SFSF equation (Eq. 1).…”

Section: Simultaneous Specular Flow and Surface Estimationmentioning

confidence: 99%

“…While most existing algorithms for specular shape reconstruction indeed assume known or calibrated illumination, recent studies have addressed the problem without such knowledge by exploiting relative motion between the observed object, the camera, or the environment. In the image plane such motion induces a specular flow -the optical flow of observed specular reflections -which has been shown to be independent of environment content and therefore to facilitate shape reconstruction (e.g., [1,2,9,17,22]). …”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Specular Flow and Shape in One Shot:

Adato

Ben-Shahar

2011

Procedings of the British Machine Vision Conference 2011

Self Cite

View full text Add to dashboard Cite

The reconstruction of specular shape from images is a very challenging task, especially when the illumination environment is unknown. In such cases, the problem becomes more tractable once the camera observes relative motion between the object and the environment, which induces a type of optical flow field that has become known as specular flow. Unfortunately, however, the estimation of specular flow from image sequences is even more challenging, putting in question the effectiveness of the shape from specular flow approach as a whole. Here we show that instead of the traditional (and somewhat futile) process of first estimating the specular flow and then using it for the recovery of specular shape, an approach that addresses these two inference problems simultaneously improves the estimation of both structures. We formulate the problem in a variational setting, we identify and address numerical issues unique to its application on specular flows, and we employ a polar representation of motion, all to result in the first ever practical method to compute specular shape from real image sequences under unknown illumination.

show abstract

On Shape and Material Recovery from Motion

Chandraker

2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We present a framework for the joint recovery of the shape and reflectance of an object with dichromatic BRDF, using motion cues. We show that four (small or differential) motions of the object, or three motions of the camera, suffice to yield a linear system that decouples shape and BRDF. The theoretical benefit is that precise limits on shape and reflectance recovery using motion cues may be derived. We show that shape may be recovered for unknown isotropic BRDF and light source. Simultaneous reflectance estimation is shown ambiguous for general isotropic BRDFs, but possible for restricted BRDFs representing commong materials like metals, plastics and paints. The practical benefit of the decoupling is that joint shape and BRDF recovery need not rely on alternating methods, or restrictive priors. Further, our theory yields conditions for the joint estimability of shape, albedo, BRDF and directional lighting using motion cues. Surprisingly, such problems are shown to be well-posed even for some non-Lambertian material types. Experiments on measured BRDFs from the MERL database validate our theory.

show abstract

A linear formulation of shape from specular flow

Cited by 28 publications

References 22 publications

Toward robust estimation of specular flow

Toward robust estimation of specular flow

Specular Flow and Shape in One Shot:

On Shape and Material Recovery from Motion

Contact Info

Product

Resources

About