Real-time visual servoing

Abstract

This paper describes a new real-time tracking algorithm in conjunction with a predictive filter to allow real-time visual servoing of a robotic arm that is following a moving object. The system consists of two calibrated (but unregistered) cameras that provide images to a real-time, pipelined-parallel optic-flow algorithm that can robustly compute optic-flow and calculate the 3-D position of a moving object at approximately 5 Hz rates. These 3-D positions of the moving object serve as input to a predictive kinematic control algorithm that uses an α − β − γ filter to update the position of a robotic arm tracking the moving object. Experimental results are presented for the tracking of a moving model train in a variety of different trajectories.

TRICLOPS: A tool for studying active vision

“…The need for prediction to compensate for image-processing delays in real-time visual servoing tasks is well known (Allen et al 1990;Brown 1989;Robinson 1987). In contrast with the c~-/3-3, filters used for prediction by Allen et al (1990) and Brown (1989), the prediction used in the current experiments is based on performing least-squares fits of low-order polynomial equations of time to a limited sequence of object position data. These motion functions are then evaluated at a time approximately equal to the estimated processing delay.…”

Mousebuster: a robot for real-time catching

“…The issue of high performance hand-eye coordination is also reported in 111, [SI, [22], where the problem of robot juggling has been considered. Grasping a moving target with a robot arm has been approached by [2], [3], [8], 1181. Allen et al split the control algorithm in two steps: a filtering and prediction phase, during which the robot tracks the object motion with the desired precision, and a catching phase, in which the robot is driven toward the target as fast as possible.…”

Acquiring visual servoing reaching and grasping skills using neural reinforcement learning

“…Here, a control signal is generated directly from the sensory input, without the need for high-level reasoning. However, even approaches capable of truly reactive operation [10], [11], [12] typically incorporate a significant amount of prior knowledge about the system and the task's solution. Common requirements range from camera calibration over the use of complete system models for generating the visual-motor Jacobian to hand-crafted control strategies [13].…”