Visual Predictive Control Strategy for Mobile Manipulators

Abstract

This work aims at designing a visual predictive control (VPC) scheme for a mobile manipulator. It consists in combining image-based visual servoing with model predictive control to benefit from the advantages of both control structures. Two challenges are addressed in this paper: the choice of the visual features and the closed-loop stability. The first ones rely on image moments to improve the end effector positioning precision. The second one is tackled through a terminal constraint coupled with suitable input constraints to reduce the computational burden. Simulation results using ROS and Gazebo validate the proposed approach.

Multi-camera Visual Predictive Control Strategy for Mobile Manipulators

“…1) The prediction models: Two prediction models f are needed, one for each camera. These latter are obtained with the same global and exact method as in [23]. Two main steps are followed.…”

“…However, these methods need a very efficient 3D reconstruction to reach an accurate pose after convergence [21]. To our knowledge, the works presented in [22] and [23] are the only ones to consider a VPC strategy to control a mobile manipulator. In [22], a point-based hierarchical MPC is used to control an underwater manipulator vehicle, but stability issues are not taken into account.…”

“…In [22], a point-based hierarchical MPC is used to control an underwater manipulator vehicle, but stability issues are not taken into account. In [23], image moments obtained from a camera mounted on the end-effector of a mobile manipulator are used to control the whole system. A terminal constraint guarantees closed-loop stability while the last velocity constraints are relaxed to ensure feasibility.…”

“…This paper thus aims at extending the work presented in [23] with the design of a multi-camera VPC strategy and its implementation on the TIAGo robotic platform. This robot is equipped with two cameras located on the head and on the arm wrist.…”

“…First, while the landmark is not visible, the visual features are projected on an image sphere [24], i.e., without projection singularities, and the whole system is controlled using cues based on spherical image moments [25]. Next, once the landmark becomes visible, the features are computed directly in the end-effector image and the robot is controlled relying on planar image moments, as these features can be more easily weighted to obtain an accurate final pose [23].…”