Automated tracking and grasping of a moving object with a robotic hand-eye system

Abstract

Most robotic grasping tasks assume a stationary or fixed object. In this paper, we explore the requirements for tracking and grasping a moving object. The focus of our work is to achieve a high level of interaction between a real-time vision system capable of tracking moving objects in 3-D and a robot arm with gripper that can be used to pick up a moving object. There is an interest in exploring the interplay of hand-eye coordination for dynamic grasping tasks such as grasping of parts on a moving conveyor system, assembly of articulated parts, or for grasping from a mobile robotic system. Coordination between an organism's sensing modalities and motor control system is a hallmark of intelligent behavior, and we are pursuing the goal of building an integrated sensing and actuation system that can operate in dynamic as opposed to static environments. The system we have built addresses three distinct problems in robotic hand-eye coordination for grasping moving objects: fast computation of 3-D motion parameters from vision, predictive control of a moving robotic arm to track a moving object, and interception and grasping. The system is able to operate at approximately human arm movement rates, and experimental results in which a moving model train is tracked is presented, stably grasped, and picked up by the system. The algorithms we have developed that relate sensing to actuation are quite general and applicable to a variety of complex robotic tasks that require visual feedback for arm and hand control.

Object tracking in videos by evolutionary clustering and locally linear neuro-fuzzy models

“…Many of previous works have been focused on tracking of human face in videos [4,5]. Others have been concerned on robotic applications of object tracking [6,7,8].…”

Real-Time Tracking of Moving Objects with an Active Camera

“…Four different control methods (LQG, pole assignment with DARMA and ARMAX models, and PI) are compared, with special emphasis on the disturbance treatment. Allen et al [27] use a stationary stereo camera system and employ object detection by thresholding the normal flow magnitude. A position prediction is based on the curvature of the trajectory and the velocity of the object.…”

Adaptive visual servoing for various kinds of robot systems

“…In most of the previous work on visual servoing, they assumed that the system structure and parameters were known [2][3][4][5][6][7], or that the parameters could be identified in an off-line process [14]. Such a controller, however, is not robust for disturbances and changes of the parameters.…”

“…For vision-based robots, image features on the image planes are primitive descriptions of the environments. In this sense, featurebased visual servoing control is the most fundamental one for the vision-based robots in which image features are controlled to converge to the desired ones, and therefore has been focused by a number of researchers [2][3][4][5][6][7][8][9][10][11][12][13].…”