Multi-Task Template Matching for Object Detection, Segmentation and Pose Estimation Using Depth Images

Park, Kiru; Patten, Timothy; Prankl, Johann; Vincze, Markus

doi:10.1109/icra.2019.8794448

Cited by 44 publications

(24 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Park et al proposed an MTTM method for object detection, segmentation, and pose estimation. In contrast to other methods that rely on the color and texture information, the MTTM method uses depth images as input and estimates the object poses using the nearest neighborhood matching [8].…”

Section: B Geometry-based Methodsmentioning

confidence: 99%

“…Thanks to the well-developed deep learning-based object detection methods, the view-based 6D target object detection methods such as [5], [6] and the view-geometry fusion method such as [7] yielded promising results on a public dataset. Similar to the RGB images, researchers represented the object geometry with the depth image and achieved satisfactory results [8].…”

Section: Introductionmentioning

confidence: 99%

“…Instead of using RGB or depth images as the input for the 6D target object detection problem, we are interested in solving this problem with the point cloud data directly for following reasons: i) the point cloud is a general representation of geometric information, which can be used to represent not only regularized 2.5D depth image but also 3D CAD models and irregular Lidar generated point-cloud data, and ii) compared to the depth image, the additional dimension of the point cloud makes it possible to segment the object out of the background without using complex methods such as a segmentation network. The permutation invariant property and sparsity of point clouds make the architectures of the well-developed image data-based 6D detection methods such as those in [5], [8] obsolete for the point cloud data. Thus, detecting object 6D pose from the point cloud data in an end-to-end fashion is an interesting problem that we have addressed in this paper.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Detecting 6D Poses of Target Objects From Cluttered Scenes by Learning to Align the Point Cloud Patches With the CAD Models

Chen

You

et al. 2020

IEEE Access

View full text Add to dashboard Cite

6D target object detection is of great importance to many applications such as robotics, industrial automation, and unmanned vehicles and is increasingly influencing broad industries including manufacturing, transportation, and retail industries, to name a few. Unlike the more common object detection methods that use the two-dimensional data such as RGB or depth images, the method proposed here relies on the three-dimensional point cloud of target objects to detect them in cluttered scenes in an end-to-end fashion. However, conventional point cloud-based 6D object detection methods rely on the robustness of key-point detection results that are not straightforward for humans to understand. The drawback makes conventional point cloud-based methods require expert knowledge to tune. In this paper, a point cloud-based 6D target object detection method that uses segmented object point cloud patches to predict object 6D poses and identity is introduced. In this way, the key-point detection is substituted with a point cloud segmentation procedure that is easier to visualize and tune. To extract the object pose and identity information from point cloud patches, we proposed an end-to-end data-driven pose correction model that can be trained with synthetic data and run on real-world data. A simple yet efficient basis spanning layer booster is proposed to accelerate the learning process and improve the pose estimation precision. An alignment error-based loss function is also introduced to make the proposed pose correction model trainable, efficiently. Experiments show that although the proposed model is trained only using object CAD models, its 6D detection performance matches that of the models using view data. Thus, the proposed method is suitable for 6D detection applications that have object CAD models instead of labeled scene data.

show abstract

Section: B Geometry-based Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detecting 6D Poses of Target Objects From Cluttered Scenes by Learning to Align the Point Cloud Patches With the CAD Models

Chen

You

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Our approach for grasping relies on first finding the nearest observation in a database and second predicting dense geometric correspondences to transform a successful grasp pose to a new observation. Retrieving similar samples has been addressed using learned feature descriptors from RGB-D images (Wohlhart and Lepetit, 2015 ; Balntas et al, 2017 ; Park et al, 2019a ). These employ the triplet loss to train a network to produce smaller feature distances for pairs of images with similar viewpoints while producing larger feature distances for pairs of images with different viewpoints or those that contain different object classes.…”

Section: Related Workmentioning

confidence: 99%

DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-Based Robotic Grasping

2020

Self Cite

View full text Add to dashboard Cite

This article presents a method for grasping novel objects by learning from experience. Successful attempts are remembered and then used to guide future grasps such that more reliable grasping is achieved over time. To transfer the learned experience to unseen objects, we introduce the dense geometric correspondence matching network (DGCM-Net). This applies metric learning to encode objects with similar geometry nearby in feature space. Retrieving relevant experience for an unseen object is thus a nearest neighbor search with the encoded feature maps. DGCM-Net also reconstructs 3D-3D correspondences using the view-dependent normalized object coordinate space to transform grasp configurations from retrieved samples to unseen objects. In comparison to baseline methods, our approach achieves an equivalent grasp success rate. However, the baselines are significantly improved when fusing the knowledge from experience with their grasp proposal strategy. Offline experiments with a grasping dataset highlight the capability to transfer grasps to new instances as well as to improve success rate over time from increasing experience. Lastly, by learning task-relevant grasps, our approach can prioritize grasp configurations that enable the functional use of objects.

show abstract

“…It features the advantages of improved efficiency and accuracy respectively in terms of learning and prediction due to the fact that commonalities and differences across tasks are exploited [33]. In addition, it is effective in the avoidance of overfitting on a specific task since the network model is regularized [34]. Wang et al managed to improve the 6D object pose estimation performance, especially under the condition of occlusion via a multitask learning network combining object recognition with pose estimation [35].…”

Section: Multitask Learningmentioning

confidence: 99%

Symmetry-Aware 6D Object Pose Estimation via Multitask Learning

Zhang

Fang

et al. 2020

Complexity

View full text Add to dashboard Cite

Although 6D object pose estimation has been intensively explored in the past decades, the performance is still not fully satisfactory, especially when it comes to symmetric objects. In this paper, we study the problem of 6D object pose estimation by leveraging the information of object symmetry. To this end, a network is proposed that predicts 6D object pose and object reflectional symmetry as well as the key points simultaneously via a multitask learning scheme. Consequently, the pose estimation is aware of and regulated by the symmetry axis and the key points of the to-be-estimated objects. Moreover, we devise an optimization function to refine the predicted 6D object pose by considering the predicted symmetry. Experiments on two datasets demonstrate that the proposed symmetry-aware approach outperforms the existing methods in terms of predicting 6D pose estimation of symmetric objects.

show abstract

Multi-Task Template Matching for Object Detection, Segmentation and Pose Estimation Using Depth Images

Cited by 44 publications

References 19 publications

Detecting 6D Poses of Target Objects From Cluttered Scenes by Learning to Align the Point Cloud Patches With the CAD Models

Detecting 6D Poses of Target Objects From Cluttered Scenes by Learning to Align the Point Cloud Patches With the CAD Models

DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-Based Robotic Grasping

Symmetry-Aware 6D Object Pose Estimation via Multitask Learning

Contact Info

Product

Resources

About