A Comprehensive Review on 3D Object Detection and 6D Pose Estimation With Deep Learning

Hoque, Sabera; Arafat, Md. Yasir; Xu, Shuxiang; Maiti, Ananda; Wei, Yuchen

doi:10.1109/access.2021.3114399

Cited by 42 publications

(12 citation statements)

References 238 publications

(304 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The two-dimensional lines corresponding to the real images of three-dimensional lines corresponding to the coplanar are also carried out by Eq. (7). Several groups of 2D-3D point pairs generated are saved and EPnP can be used for pose registration.…”

Section: C) Matching Of Multiple Edge Featuresmentioning

confidence: 99%

See 1 more Smart Citation

Reflective texture-less object registration using multiple edge features for augmented reality assembly

Zhao²,

Zhao³

et al. 2022

Int J Adv Manuf Technol

View full text Add to dashboard Cite

Virtual assembly technology has developed rapidly in recent years. As one of its key technologies, object registration has been widely studied, whose main task is to register the target object. However, the existing methods cannot effectively register re ective texture-less objects such as metal parts. In this paper, an object registration method based on multiple edge features is proposed, which includes a sparse uniform template sampling method and a multiple edge feature matching method. At the same time, we also used two oil pumps to carry out virtual assembly experiments and verify the accuracy of object registration. The experimental results show that this method can achieve high accuracy which is in line with the requirements of the virtual assembly algorithm in the industry.

show abstract

Section: C) Matching Of Multiple Edge Featuresmentioning

confidence: 99%

“…According to the image type used, object registration methods can be divided into methods based on three-dimensional information (3D point cloud) and methods based on two-dimensional information (2D image). In recent years, in both 2D and 3D image-based object registration, deep learning becomes a useful tool [7,8].…”

Section: Introductionmentioning

confidence: 99%

Reflective texture-less object registration using multiple edge features for augmented reality assembly

Zhao²,

Zhao³

et al. 2022

Int J Adv Manuf Technol

View full text Add to dashboard Cite

show abstract

“…Most state-of-the-art methods [1,2,3,4,5,6,7,8,9,10,11,12] for 3D pose estimation typically require object's 3D model or 2D-3D correspondence information. Approaches that rely on 2D-3D correspondence estimate pixel-wise dense correspondence [4,5,9], or matching between a number of sparse keypoints [6,10].…”

Section: Introductionmentioning

confidence: 99%

CVML-Pose: Convolutional VAE Based Multi-Level Network for Object 3D Pose Estimation

2023

View full text Add to dashboard Cite

Most vision-based 3D pose estimation approaches typically rely on knowledge of object's 3D model, depth measurements, and often require time-consuming iterative refinement to improve accuracy. However, these can be seen as limiting factors for broader real-life applications. The main motivation for this paper is to address these limitations. To solve this, a novel Convolutional Variational Auto-Encoder based Multi-Level Network for object 3D pose estimation (CVML-Pose) method is proposed. Unlike most other methods, the proposed CVML-Pose implicitly learns an object's 3D pose from only RGB images encoded in its latent space without knowing the object's 3D model, depth information, or performing a post-refinement. CVML-Pose consists of two main modules: (i) CVML-AE representing convolutional variational autoencoder, whose role is to extract features from RGB images, (ii) Multi-Layer Perceptron and K-Nearest Neighbor regressors mapping the latent variables to object 3D pose including, respectively, rotation and translation. The proposed CVML-Pose has been evaluated on the LineMod and LineMod-Occlusion benchmark datasets. It has been shown to outperform other methods based on latent representations and achieves comparable results to the state-of-the-art, but without use of a 3D model or depth measurements. Utilizing the t-Distributed Stochastic Neighbor Embedding algorithm, the CVML-Pose latent space is shown to successfully represent objects' category and topology. This opens up a prospect of integrated estimation of pose and other attributes (possibly also including surface finish or shape variations), which, with real-time processing due to the absence of iterative refinement, can facilitate various robotic applications. Code available: https://github.com/JZhao12/CVML-Pose.

show abstract

“…To accomplish this, there is the need to develop a robust algorithm capable of estimating the object's pose inside the 360°frame. 6DoF estimation is one of the main challenging research topics in computer vision [12][13][14].…”

Section: Introductionmentioning

confidence: 99%

Object Pose Detection to Enable 3D Interaction from 2D Equirectangular Images in Mixed Reality Educational Settings

et al. 2022

View full text Add to dashboard Cite

In this paper, we address the challenge of estimating the 6DoF pose of objects in 2D equirectangular images. This solution allows the transition to the objects’ 3D model from their current pose. In particular, it finds application in the educational use of 360° videos, where it enhances the learning experience of students by making it more engaging and immersive due to the possible interaction with 3D virtual models. We developed a general approach usable for any object and shape. The only requirement is to have an accurate CAD model, even without textures of the item, whose pose must be estimated. The developed pipeline has two main steps: vehicle segmentation from the image background and estimation of the vehicle pose. To accomplish the first task, we used deep learning methods, while for the second, we developed a 360° camera simulator in Unity to generate synthetic equirectangular images used for comparison. We conducted our tests using a miniature truck model whose CAD was at our disposal. The developed algorithm was tested using a metrological analysis applied to real data. The results showed a mean difference of 1.5 with a standard deviation of 1 from the ground truth data for rotations, and 1.4 cm with a standard deviation of 1.5 cm for translations over a research range of ±20 and ±20 cm, respectively.

show abstract

A Comprehensive Review on 3D Object Detection and 6D Pose Estimation With Deep Learning

Cited by 42 publications

References 238 publications

Reflective texture-less object registration using multiple edge features for augmented reality assembly

Reflective texture-less object registration using multiple edge features for augmented reality assembly

CVML-Pose: Convolutional VAE Based Multi-Level Network for Object 3D Pose Estimation

Object Pose Detection to Enable 3D Interaction from 2D Equirectangular Images in Mixed Reality Educational Settings

Contact Info

Product

Resources

About