Visual Servoing/Tracking Using Central Catadioptric Images

Abstract

International audienceVisual control of robot motion may benefit from enhanced camera field of view. With traditional cameras the available fields of view are only enough to view a region around the observed object (for eye-in-hand systems) or around the end-effector (for independent-eye systems). Central catadioptric systems have larger fields of view thus allowing the entire robot AND the surrounding objects to be imaged with a unique camera. Therefore, the whole robot's articulated mechanism can be observed and its joints can be tracked and controlled simultaneously. This results in a new visual robot control concept where tracking and control are embedded together. Key to the understanding of both servoing and tracking is the central catadioptric Jacobian matrix linking the robot's joint velocities to image observations. In spite of a more complex projection matrix associated with catadioptric sensors, we study the catadioptric Jacobian matrix and we show that it does not introduce any additional singularity with respect to the traditional pinhole camera model. Experiments showing a rigid body being tracked with a catadioptric camera are described

Coarsely calibrated visual servoing of a mobile robot using a catadioptric vision system

“…In addition, for the estimation of P (see (14)), the value of |P z | has been set to the constant value 0.8 m which roughly corresponds to the desired depth |P * z |. Except for one particular experiment, the value of the gain of the control (6) has been set to λ = 0.1.…”

“…However the corresponding interaction matrix, obtained from the six DOFs case in [14], is highly coupled:…”

Fitting 3D Models on Central Catadioptric Images

“…In [18] an SSD-based tracker allows the tracking of planar structure using an efficient second order minimization method. Closer to our problem is [3] where a model-based tracker based on a global non-linear minimization is presented. Although the goal is very similar, the modeling of the cost function as well as the minimization issue that we consider in this paper are different.…”

“…These 2D-3D registration techniques rely on the use of a 3D model of the tracked objects. Considering catadioptric camera, Authors are with INRIA, IRISA, Lagadic, F-35000 Rennes, France ; email marchand@irisa.fr similar approaches can be considered [3]. Nevertheless, a new projection model has to be considered which implies the definition of new visual features and of new Jacobians to be used in the minimization approach.…”

Visual Servoing from Spheres with Paracatadioptric Cameras

“…Considering feature points on such cameras, the interaction with the system (the link between the robot velocities and the image observations) has been shown to present the same singularities as classical perspective cameras [9]. Lately, a spherical projection model has been used to design a new minimal set of optimal visual features for visual servoing from spheres with any central catadioptric system [10].…”