3D object instance recognition and pose estimation using triplet loss with dynamic margin

Abstract

In this paper, we address the problem of 3D object instance recognition and pose estimation of localized objects in cluttered environments using convolutional neural networks. Inspired by the descriptor learning approach of Wohlhart et al.[1], we propose a method that introduces the dynamic margin in the manifold learning triplet loss function. Such a loss function is designed to map images of different objects under different poses to a lower-dimensional, similarity-preserving descriptor space on which efficient nearest neighbor search algorithms can be applied. Introducing the dynamic margin allows for faster training times and better accuracy of the resulting low-dimensional manifolds. Furthermore, we contribute the following: adding in-plane rotations (ignored by the baseline method) to the training, proposing new background noise types that help to better mimic realistic scenarios and improve accuracy with respect to clutter, adding surface normals as another powerful image modality representing an object surface leading to better performance than merely depth, and finally implementing an efficient online batch generation that allows for better variability during the training phase.We perform an exhaustive evaluation to demonstrate the effects of our contributions. Additionally, we assess the performance of the algorithm on the large BigBIRD dataset [2] to demonstrate good scalability properties of the pipeline with respect to the number of models.

The RGB-D Triathlon: Towards Agile Visual Toolboxes for Robots

“…For the LineMOD dataset [38] we consider two state of the art approaches [45], [43]. In [43] a convolutional network is used to map the image space to a descriptor space where the pose and object classes are predicted through a nearest neighbour classifier.…”

“…In [43] a convolutional network is used to map the image space to a descriptor space where the pose and object classes are predicted through a nearest neighbour classifier. The method in [45] builds upon [43], introducing a triplet loss function with a dynamic margin. These works employ a slightly different settings than ours since they use synthetic images.…”

“…Some methods tested on this dataset consider the use of both synthetic and real images [45], [46]. In this paper, in order to have a simpler experimental setup, we follow [47] and we consider a setting where only real images are used for training and testing the models.…”

When Regression Meets Manifold Learning for Object Recognition and Pose Estimation

“…pulling viewpoints under similar poses close together and pushing dissimilar ones or different objects further away. As appeared in [13], m corresponds to a dynamic margin defined as:…”

“…To analyze not only our method, but the effect of multitask learning, i.e. regression and learning robust feature descriptors together, we report the results compared to the baseline method [13]. Here we train on the loss function L d to compare to the results obtained by nearest neighbor pose retrieval, abbreviated as NN.…”