MaskUKF: An Instance Segmentation Aided Unscented Kalman Filter for 6D Object Pose and Velocity Tracking

Abstract

Tracking the 6D pose and velocity of objects represents a fundamental requirement for modern robotics manipulation tasks. This paper proposes a 6D object pose tracking algorithm, called MaskUKF, that combines deep object segmentation networks and depth information with a serial Unscented Kalman Filter to track the pose and the velocity of an object in real-time. MaskUKF achieves and in most cases surpasses state-of-the-art performance on the YCB-Video pose estimation benchmark without the need for expensive ground truth pose annotations at training time. Closed loop control experiments on the iCub humanoid platform in simulation show that joint pose and velocity tracking helps achieving higher precision and reliability than with one-shot deep pose estimation networks. A video of the experiments is available as Supplementary Material.

TP-AE: Temporally Primed 6D Object Pose Tracking with Auto-Encoders

“…However, focusing on only two consecutive steps is restrictive in fully characterizing objects' motion. To this end, [4], [31] use Rao-Blackwellized Particle Filter [32] and Unscented Kalman Filter [33], respectively, to encode motion information that is subsequently used for object tracking. Nonetheless, the performance of [4] under occlusion remains unsatisfying due to the lack of utilization of temporal information in the object reconstruction phase.…”

An Improved Approach to 6D Object Pose Tracking in Fast Motion Scenarios

“…A follow up work [29] leverages real-time optical flow to improve tracking performance in fast motion. In [27], for each RGB-D frame, the segmentation mask of the object of interest is combined with the depth frame to produce a partial point cloud of the object which is further refined by outlier removal. The refined measurements are fed into an Unscented Kalman Filter to correct the belief of the object pose and velocity.…”

“…2) Process model: Following [27], we use the White Noise Acceleration model (WNA) [31], which assumes that the Cartesian acceleration is driven by a white noise process w t and the angular acceleration is driven by a white noise process w r . The state equations are given by…”

Robot Bionic Vision Technologies: A Review

“…For example, one of the features of the iCub robot is that it is equipped with a stereo camera rig in the head and a 320 × 240 resolution color camera in each eye. This setup consists of three actuated DoF in the neck of the robot to grant the roll, pitch, and yaw capabilities to the head, as well as three actuated DoF to model the oculomotor system of a human being (tilt, version, and vergence) [88]. Furthermore, F. Bottarel [89] made a comparison of the Mask R-CNN and Faster R-CNN algorithms using the bionic vision hardware of the iCub robot [90] and discovered that Mask R-CNN is superior to Faster R-CNN.…”