Multi-Camera Active-Vision for Markerless Shape Recovery of Unknown Deforming Objects

Abstract

This thesis proposes a multi-camera active-vision reconfiguration system which selects camera poses online to improve the shape recovery of a priori unknown, markerless, deforming objects in dynamic environments. The objectives of shape recovery are defined as surface sampling accuracy, and shape completeness. The completeness objective is generalized for both solid and surface-based objects as the maximization of surface visibility. Thus, improving the recovered shape of target objects is shown to be analogous to maximizing the surface area visibility. This improvement is achieved through the on-line reconfiguration of multiple cameras. The system developed herein is composed of a shape recovery method, a robust tracking algorithm, and a multi-camera reconfiguration method.The shape recovery method is based on a modular fusion technique that produces a complete 3D mesh-model of the target object. The method fuses triangulation data with a visual hull to maximize recovery accuracy. The modularity of the method lies in the ability to modify data filtering techniques to improve modelling accuracy, and interchange stereo correspondence features. The adaptive particle filtering algorithm produces a deformation estimation of the recovered model from the tracking data. The algorithm automatically adapts to the quantity of tracking data available and changes in the object's dynamics. The modularity of the algorithm allows modifications in terms of the number of particles and motion models for task-specific implementations. The reconfiguration method consists of a robust stereovisibility objective function, workspace discretization, and a path planner. The complete system can iii improve the shape recovery of a priori unknown deforming objects in obstacle-laden environments when compared to static camera methods.To validate the proposed system, extensive simulations and experiments were conducted. The simulations tested the system in comparison to static multi-camera systems, and ideal camera placement where the object model was a priori known and the camera's dynamics were unconstrained. Simulation results showed the proposed methodology outperformed static cameras and approached the performance of ideal camera placement in a dynamic, obstacle-laden environment. The experimental results showed similar improvement in shape recovery when comparison to a static camera system in an obstacle-laden environment.iv

Coverage of deformable contour shapes with minimal multi-camera system

“…The minimum number of cameras can be obtained with additional formulation from the setup described in Proposition 1. Approaches dealing with coverage problems often include a visibility map or visibility function, which relates the point of view of a sensor with the parts of the object it is covering at certain time instant [15], [28]. From Proposition 1 we can also consider a visibility matrix V S×C = [V ji ] that indicates which segments are detected by each camera, i.e.…”

“…As for the active perception field [25], [26], works in this domain are dynamic in general. Multi-camera centralized networks, where the position and orientation of each sensor are optimized, are considered in studies of this field for shape recovery of moving deformable objects [27], [28]. Compared to our approach, these works do not minimize the number of cameras that are necessary to recover the shape of the object.…”

Coverage Optimization of Camera Network for Continuous Deformable Object

“…For instance, for the parts in machining process, it needs to deploy the cameras for monitoring the whole machining process. This kind of problem can be regarded as visual coverage for deformable object, which has been discussed in [13]- [15] via dynamical deploying the cameras according to the deformation process. Due to the field of view for the camera is always wide, the static deployment of cameras is sufficient for visual coverage on deformable object.…”

Multi-camera coverage of deformable contour shapes

“…Works in the active perception field are usually dynamic in the sense that in order to get more information about the environment, whose properties may change over time, the sensors must reconfigure their position and/or orientation depending on the requirements of the perception task. The multi-camera centralized networks in [15] and [16] are optimized in position and orientation so that they are able to recover the shape of a moving and deformable target object, but they do not consider robust perception aspects such as occlusion avoidance.…”