3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

Abstract

Inspired by the recent success of methods that employ shape priors to achieve robust 3D reconstructions, we propose a novel recurrent neural network architecture that we call the 3D Recurrent Reconstruction Neural Network (3D-R2N2). The network learns a mapping from images of objects to their underlying 3D shapes from a large collection of synthetic data [1]. Our network takes in one or more images of an object instance from arbitrary viewpoints and outputs a reconstruction of the object in the form of a 3D occupancy grid. Unlike most of the previous works, our network does not require any image annotations or object class labels for training or testing. Our extensive experimental analysis shows that our reconstruction framework i) outperforms the state-of-theart methods for single view reconstruction, and ii) enables the 3D reconstruction of objects in situations when traditional SFM/SLAM methods fail (because of lack of texture and/or wide baseline).

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

“…Given a single image, they use a neural network to predict the underlying 3D object as a 3D volume. There are two key differences between our work and [5]: First, the predicted object in [5] is a 3D volume; whilst ours is a point cloud. As demonstrated and analyzed in Sec 5.2, point set forms a nicer shape space for neural networks, thus the predicted shapes tend to be more complete and natural.…”

“…More relevant to our work is [5]. Given a single image, they use a neural network to predict the underlying 3D object as a 3D volume.…”

“…Concretely we used a subset of 220K models covering 2,000 object categories. The use of synthesized data has been adopted in a number of existing works [5,19].…”

“…Comparison to state-of-the-art We compare our work to 3D-R2N2 [5] which is the state-of-the-art in deep learning based 3D object generation. 3D-R2N2 reconstructs 3D from single or multi-view images into a volumetric representation.…”

Preparation of Translated, Scaled, and Rotated ATS Drugs 3D Molecular Structure for the Validation of 3D Moment Invariants-based Molecular Descriptors

“…However, since there is currently no repository or database available which provide the imaging results of nc-AFM and the existing model of 3D molecular structure is similar to 3D conformation of molecular compounds [30], this paper alternatively generates the same output file format produced by 3D object reconstruction algorithm, which is binary volume pixel (voxel) grid [55,56] to simulate the drugs molecular structure construction process obtained using nc-AFM. The detailed procedure on how this paper generates these files will be discussed in Section 3.…”