Binding Computer Vision to Physics Based Simulation: The Case Study of a Bouncing Ball

Abstract

A dynamic scene and, therefore, its visual observations are invariably determined by the laws of physics. We demonstrate an illustrative case where physical explanation, as a vision prior, is not a commodity but a necessity. By considering the problem of ball motion estimation we show how physics-based simulation in conjunction with visual processes can lead to the reduction of the visual input required to infer physical attributes of the observed world. Even further, we show that the proposed methodology manages to reveal certain physical attributes of the observed scene that are difficult or even impossible to extract by other means. A series of experiments on synthetic data as well as experiments with image sequences of an actual ball, support the validity of the proposed approach. The use of generic tools and the top-down nature of the proposed approach make it general enough to be a likely candidate for handling even more complex problems in larger contexts.

Grasping Grasping in GRASP

“…5. Another new research area was the work on the formulation of a method for inferring physical attributes of the observed world with the aid of physics based simulation [15]. Here, a full model of a human hand was developed and integrated in the simulator [20].…”

Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis

“…Duff and Wyatt [8] used physical simulation and search heuristics to track a fast moving ball, despite occlusions and for the 2D case. In previous work [13], we performed 3D motion estimation for a bouncing ball, from a single camera and despite severe occlusions by exploiting dynamics modelling. Ye and Liu [26] synthesized physically plausible hand movements, from pour or absent hand observations, that explained the manipulation of objects with known trajectories from a hand whose rough location was also known.…”

A computational framework for observing and understanding the interaction of humans with objects of their environment

“…5.3), as two different sensor setups. Over these setups three variants of the 3D hand tracking problem are formulated, that correspond to works [52,61,70] (see fig. 5.1, fig.…”

“…With respect to [67], the objective function is identical to the one in [52], the only difference being the decoder (an object decoder was swapped for another hand decoder) and the fact that the finger interpenetration penalty needs to be computed twice (once for each hand). Interestingly, adding a term that penalized inter-hand penetrations was not required, as ambiguities in observations could already be resolved.…”

“…Duff and Wyatt [33] used physical simulation and search heuristics to track a fast moving ball, despite occlusions and for the 2D case. In previous work [52], we performed 3D motion estimation for a bouncing ball, from a single camera and despite severe occlusions by exploiting dynamics modelling. Ye and Liu [116] synthesized physically plausible hand movements, from pour or absent hand observations, that explained the manipulation of objects with known trajectories from a hand whose rough location was also known.…”