Single View 3D Reconstruction Based on Improved RGB-D Image

Cao, Mingwei; Zheng, Liping; Liu, Xiaoping

doi:10.1109/jsen.2020.2968477

Cited by 13 publications

(7 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some methods suggest learning a 3D shape model from key points or silhouettes [29]. In some studies, the single image's depth map is first calculated using machine-learning-based techniques, and then a 3D model is constructed using RGB-D images [30].…”

Section: Learning-based Reconstructionmentioning

confidence: 99%

Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders

2021

View full text Add to dashboard Cite

In recent years, learning-based approaches for 3D reconstruction have gained much popularity due to their encouraging results. However, unlike 2D images, 3D cannot be represented in its canonical form to make it computationally lean and memory-efficient. Moreover, the generation of a 3D model directly from a single 2D image is even more challenging due to the limited details available from the image for 3D reconstruction. Existing learning-based techniques still lack the desired resolution, efficiency, and smoothness of the 3D models required for many practical applications. In this paper, we propose voxel-based 3D object reconstruction (V3DOR) from a single 2D image for better accuracy, one using autoencoders (AE) and another using variational autoencoders (VAE). The encoder part of both models is used to learn suitable compressed latent representation from a single 2D image, and a decoder generates a corresponding 3D model. Our contribution is twofold. First, to the best of the authors’ knowledge, it is the first time that variational autoencoders (VAE) have been employed for the 3D reconstruction problem. Second, the proposed models extract a discriminative set of features and generate a smoother and high-resolution 3D model. To evaluate the efficacy of the proposed method, experiments have been conducted on a benchmark ShapeNet data set. The results confirm that the proposed method outperforms state-of-the-art methods.

show abstract

Section: Learning-based Reconstructionmentioning

confidence: 99%

Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders

2021

View full text Add to dashboard Cite

show abstract

“…u init i is the 2D projection point of 3D point p i ðx; y; zÞ according to T j ¼ ½Rjt using where l is a scale factor, R and t are the rotation matrix and translation vector that define the camera pose, and K is the camera intrinsic matrix. 21 We designed a tiny fully convolutional image patch matching network named M_R_Net to find the refined position u refined i of u init i (shown in Figure 4). In M_R_Net architecture, image patches x and z are passed to network to get the output, which is a map of matching score, the Convolutional Neural Network (CNN) blocks in the branch of x and z are shared weights, the output of the CNN block undergoes a layer of 2D convolution to obtain the final output, the position with the highest score in the matching score map represents the best matching position of z in x.…”

Section: Image Patch Matchingmentioning

confidence: 99%

A patch-based real-time six degrees of freedom object pose refinement method for robotic manipulation

Yang,

Jiang,

et al. 2024

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

A fundamental vision technique for industrial robots involves the six degrees of freedom pose estimation of target objects from a single image. However, the direct estimation of the six degrees of freedom object pose solely from a single image is subject to limited accuracy. Various refinement approaches have been proposed to improve the accuracy by utilizing rendered images from a 3D model. Nevertheless, balancing speed and accuracy in an industrial setting remains a challenge for these methods. In this study, we propose a novel six degrees of freedom pose refinement approach centered around matching real image patches. Unlike previous approaches, our method does not rely on a 3D model, resulting in increased speed and accuracy. In the offline phase, we construct an offline database using image patches obtained from real images. During the inference phase, our method initially identifies the image patch within the offline database that is closest to the initial pose. Subsequently, we refine the six degrees of freedom pose by matching the corresponding image patches from the offline database. Experimental results indicate that our six degrees of freedom pose refinement method achieves real-time capability with a frame rate of 71 Frames Per Second (FPS), along with high precision. When the threshold is set to 0.5% of the object diameter, the average distance of dots score on the test data surpasses 70%. Moreover, experiments involving gripping and assembling tasks on an industrial robot demonstrate the ability of our method to autonomously select appropriate grasping angles and positions in real time. It further generates suitable motion paths, ultimately ensuring production efficiency.

show abstract

“…(5) en, formula ( 14) is used to move the points in the feature-rich region normally to obtain the filtered new point cloud data; (6) After all points are calculated in turn and new point cloud data are obtained through transformation, end. [31]. If the data sequence is discrete, it is called a discrete time series.…”

Section: Point Cloud Smoothness In Feature-rich Regionsmentioning

confidence: 99%

CAD-Aided 3D Reconstruction of Intelligent Manufacturing Image Based on Time Series

Zhang

Wang

et al. 2022

Scientific Programming

View full text Add to dashboard Cite

To improve the three-dimensional (3D) reconstruction effect of intelligent manufacturing image and reduce the reconstruction time, a new CAD-aided 3D reconstruction of intelligent manufacturing image based on time series was proposed. Kinect sensor is used to collect depth image data and convert it into 3D point cloud coordinates. The collected point cloud data are divided into regions, and different point cloud denoising algorithms are used to filter and denoise the divided regions. With the help of CAD, FLANN matching algorithm is used to extract feature points of time-series images and complete image matching. Three-dimensional reconstruction of sparse point cloud and dense point cloud is carried out to complete 3D reconstruction of intelligent manufacturing images. The experimental results show that the image PSNR of this method is always above 52 dB, and the maximum reconstruction time is 4.9 s. The 3D reconstruction effect of intelligent manufacturing image is better, and it has higher practical application value.

show abstract

Single View 3D Reconstruction Based on Improved RGB-D Image

Cited by 13 publications

References 41 publications

Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders

Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders

A patch-based real-time six degrees of freedom object pose refinement method for robotic manipulation

CAD-Aided 3D Reconstruction of Intelligent Manufacturing Image Based on Time Series

Contact Info

Product

Resources

About