Passivity‐based adaptive 3D visual servoing without depth and image velocity measurements for uncertain robot manipulators

Abstract

In this work, we consider the 3D visual tracking problem for a robot manipulator with uncertainties in the kinematic and dynamic models. The visual feedback is provided by a fixed and uncalibrated camera located above the robot workspace. The Cartesian motion of the robot end effector can be separated into a 1D motion parallel to the optical axis of the camera and a 2D motion constrained on a plane orthogonal to this axis. Thus, the control design can be simplified, and the overall visual servoing system can be partitioned in two almost-independent subsystems. Adaptive visual servoing schemes, based on a kinematic approach, are developed for image-based look-and-move systems allowing for both depth and planar tracking of a reference trajectory, without using image velocity and depth measurements. In order to include the uncertain robot kinematics and dynamics in the presented solution, we develop a cascade control strategy based on an indirect/direct adaptive method. The stability and convergence properties are analyzed in terms of Lyapunovlike functions and the passivity-based formalism. Numerical simulations including hardware-in-the-loop results, obtained with a robot manipulator and a web camera, are presented to illustrate the performance and feasibility of the proposed control scheme. and, therefore, the control system could exhibit degraded performance particularly when the robot is driven by direct-drive actuators or it has to perform high-speed tasks. Some exceptions can be found in earlier published works [16][17][18]. However, the proposals are applied only to the planar visual servoing problem, and their robot motion controllers based on the dynamic model require the measurement of the image velocity and, thus, the need of using an extra sensor. In addition, the measurement of the velocity for a moving target object from a sequence of images is impaired by the noisy image data, affecting the accuracy of the result [19,20].In this context, adaptive schemes for camera calibration without using visual velocity were proposed for 2D image-based visual servoing systems, with global stability properties under the assumption of exact knowledge of the robot mechanical parameters [21,22]. Adaptive control strategies for 2D visual tracking with uncertain robot manipulators were also developed, where the estimated Jacobian matrix of the camera-robot systems parameters has been simultaneously updated in an indirect manner [23]. Likewise, the uncertain Jacobian matrices related to the robot kinematics and the camera calibration parameters could be independently updated by using an indirect/direct adaptive schemes, respectively [24].Conversely, the robot motion control problem in a 3D Cartesian coordinate system has been addressed from the development of a 2-1/2-D visual servoing approach by using different choices of camera architectures [25,26]. As a restriction, some of the proposed solutions require the direct estimate of the depth information with respect to the camera frame, resorting to an off-line learning p...

Delay‐Dependent Stability in Uncalibrated Image‐Based Dynamic Visual Servoing Robotic System

“…By Property 1, the Jacobian matrix D can be expressed in a linear form: a known matrix (regressor matrix) multiplies an unknown vector. From (12), it can be clearly seen that the regressor matrix Y D y, q, q includes the current image position y t . Unfortunately, the feedback visual signals are delayed as we consider.…”

“…Instead, we can only obtain Y D y t − h , q, q . After substituting this regressor matrix including the delayed visual feedback matrix into (14) and (12), we have…”

“…Some work [6][7][8] investigates the approaches with robust controllers for eliminating the negative effects of calibration errors of the system model, and the uncertain system parameters in the above work are replaced with the approximated ones. As for the case of unknown parameters or time-varying parameters, adaptive control techniques are proposed [11,12]. To handle such parameters, in these methods, adaptive laws are designed to update them online.…”

From adaptive control to variable structure systems – seeking harmony

“…In the work by Leite and Lizarralde, the 3D visual tracking problem is considered for a robot manipulator with uncertainties in the kinematic and dynamic models. The visual feedback is provided by a fixed and uncalibrated camera located above the robot workspace.…”

A Robust Visual Servoing Approach for Robotic Fruit Harvesting

“…Leite et al consider a 3D visual tracking problem with uncertainties at the kinematics and dynamics model of the robot arm, and intrinsic camera parameters, they introduce an adaptive visual servoing scheme based on a kinematic approach for an image-based-look-and-move system to perform a 3D tracking task [34].…”