A Sparse Gaussian Approach to Region-Based 6DoF Object Tracking

Abstract

In many applications of advanced robotic manipulation, six degrees of freedom (6DoF) object pose estimates are continuously required. In this work, we develop a multimodality tracker that fuses information from visual appearance and geometry to estimate object poses. The algorithm extends our previous method ICG, which uses geometry, to additionally consider surface appearance. In general, object surfaces contain local characteristics from text, graphics, and patterns, as well as global differences from distinct materials and colors. To incorporate this visual information, two modalities are developed. For local characteristics, keypoint features are used to minimize distances between points from keyframes and the current image. For global differences, a novel region approach is developed that considers multiple regions on the object surface. In addition, it allows the modeling of external geometries. Experiments on the YCB-Video and OPT datasets demonstrate that our approach ICG+ performs best on both datasets, outperforming both conventional and deep learningbased methods. At the same time, the algorithm is highly efficient and runs at more than 300 Hz. The source code of our tracker is publicly available.

TsFPS

“…Region-based methods are based a foreground-background statistical algorithm model that focuses on the statistical properties of element regions in an image, aims at differentiating silhouette of the target object from the input image. The pose can be obtained by maximizing the discrepancy between the foreground and background region statistics [18,22,25]. Early region-based methods are not computationally efficient, hard to satisfy the demands of real-time applications [5,21], till Prisacariu and Reid proopse PWP3D [18], the first real-time capable region-based method.…”

“…Hexner and Hagege [12] propose a localized appearance model framework to improve the performance of PWP3D in complex scenes, this idea is later enchanced by Tjaden et al [24,25], local color histograms are introduced to enforce temporal consistency, which improves the robustness of the tracking in the condition of occlusions, illumination changes such environmental variations. Stoiber et al [22] suggest a sparse approach based on correspondence lines, proved its efficiency both in terms of runtime and tracking success. However, due to the fact that the statistical model is based on 2D silhouette only, all the region-based methods struggle with estimating the pose of symmetrical objects that exactly the platonic solids are.…”

“…As a common method in many works [18,22,24,25], a level-set function Φ(x) is employed as the shape embedding function. There is a underlying 3D model of the platonic solid to generate 2D binary silhouette image I 𝑠 , the contour curve C of I 𝑠 can be interpreted as:…”

“…Markerless and marker-based methods are two main directions of vision-based tracking [19], both have been making significant achievements. In particular, among recent related works, numerous are based on the markerless methods that can be classified into point-based methods, region-based methods, direct methods and learning-based methods [10,13,14,19,22,24,25,28,32]. These methods rely on feature-detection algorithms to obtain specific features, which means the feature-detection has a strong impact on the final tracking performance.…”

se(3)-TrackNet: Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains

“…Temporal tracking of object poses over sequences of images can greatly improve speed while maintain or even improve pose quality [14,20,21,27,29]. Nevertheless, many dataset according to the area under the curve (AUC) metric for the ADD and ADD-S objectives [30].…”