Real-time pose tracking of 3D targets by photo-model-based stereo-vision

Abstract

Nowadays, robots have been utilized to perform a wide variety of tasks instead of humans, in which those equipped with hand-eye cameras have been proved to be useful in factories for handling metal parts. However, there have been difficulties for robots with vision sensors like cameras to detect a target objects' 3D pose accurately, especially in the case that the target object is unique and the shape is arbitrary, and further, the object is moving. Visual servoing to moving target may enable the robot to pursue an animal to catch it. Aiming at achieving identification ability of a unique object, this paper utilized our previous study concerning photo-model-based object recognition. Based on the study, this paper applies the photo-model-based pose estimation method to video-rate stereo-vision position and orientation (pose) tracking, and it has been applied to an arbitrary-shaped moving target. To confirm the abilities of the proposed stereo-vision-based method to pursue an arbitrary target, real-time 3D pose estimation experiment to track the relative pose between a moving target and hand-eye robot have been conducted. The realtime experimental results show the proposed system can recognize the designated arbitrary-shaped target and track its pose in real-time.

Expanded photo-model-based stereo vision pose estimation using a shooting distance unknown photo

“…We choose the genetic algorithm (GA) as an optimization method to find the maximum fitness value because of its simplicity and effectiveness. 31,41 Because of the limited space, GA will not be introduced in detail here.…”

“…In previous studies, 31,41,42 the chromosome in the GA consisted of six variables, representing possible pose solutions. However, for PM ratio detection, as shown in equation (16), each chromosome is elongated and now comprises seven variables.…”

“…Additionally, a 3D target object's pose could be estimated and tracked in real time by using stereo vision and its 2D photo. 31 Moreover, a visual servoing system for catching marine creatures was developed using the photo-model approach. 24 However, previous algorithms based on photo-models rely on camera calibration to obtain the pixel per metric (PM) ratio and calculate the target size based on the shooting distance.…”

“…Additionally, a 3D target object’s pose could be estimated and tracked in real time by using stereo vision and its 2D photo. 31 Moreover, a visual servoing system for catching marine creatures was developed using the photo-model approach. 24…”

Crucian carp position and pose recognition based on an improved photo-model method

“…Different from locating the target based on parallax, this method realizes target recognition and location by comparing the difference in pixel color information in the image. Tian et al [6] applied the position and pose estimation method by photo-model-based stereo-vision position and pose tracking of video rate, which realizes the position and pose estimation of various marine organisms. For the purpose of solving the problem of recognizing freshwater fish's pose and position, the photo-modelbased algorithm is improved in this paper.…”