Front2Back: Single View 3D Shape Reconstruction via Front to Back Prediction

Yao, Yuan; Schertler, Nico; Rosales, Enrique; Rhodin, Helge; Sigal, Leonid; Sheffer, Alla

doi:10.1109/cvpr42600.2020.00061

Cited by 39 publications

(21 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to include symmetric prior inside a network, we have to know the symmetry correspondence of the 3D output. To find the symmetry correspondences in view-centric coordinates, symmetry plane detection is essential [41]. In a view-centric coordinate, the global reflection symmetry plane of an object comes to vary along with the change of input view.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…To find the symmetry correspondence, these methods need either camera pose or symmetry plane as given inputs. References [40,41] proposes symmetry detection methods for 3D reconstruction. On the other hand, reference [39] propose an implicit function for 3D reconstruction and find the local details by projecting 3D points onto 2D image and applying symmetry fusion.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sym3DNet: Symmetric 3D Prior Network for Single-View 3D Reconstruction

Siddique

Lee

2022

Sensors

View full text Add to dashboard Cite

The three-dimensional (3D) symmetry shape plays a critical role in the reconstruction and recognition of 3D objects under occlusion or partial viewpoint observation. Symmetry structure prior is particularly useful in recovering missing or unseen parts of an object. In this work, we propose Sym3DNet for single-view 3D reconstruction, which employs a three-dimensional reflection symmetry structure prior of an object. More specifically, Sym3DNet includes 2D-to-3D encoder-decoder networks followed by a symmetry fusion step and multi-level perceptual loss. The symmetry fusion step builds flipped and overlapped 3D shapes that are fed to a 3D shape encoder to calculate the multi-level perceptual loss. Perceptual loss calculated in different feature spaces counts on not only voxel-wise shape symmetry but also on the overall global symmetry shape of an object. Experimental evaluations are conducted on both large-scale synthetic 3D data (ShapeNet) and real-world 3D data (Pix3D). The proposed method outperforms state-of-the-art approaches in terms of efficiency and accuracy on both synthetic and real-world datasets. To demonstrate the generalization ability of our approach, we conduct an experiment with unseen category samples of ShapeNet, exhibiting promising reconstruction results as well.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Sym3DNet: Symmetric 3D Prior Network for Single-View 3D Reconstruction

Siddique

Lee

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…A very few works in the literature look into cameraframe object reconstructions. Yao et al [24] presented a method which estimates the object symmetry plane and predicts front and back orthographic views. Their method, trained separately for every new object class, uses a GAN component which requires a large number of images for training.…”

Section: Related Workmentioning

confidence: 99%

Object Shell Reconstruction: Camera-centric Object Representation for Robotic Grasping

Chavan-Dafle¹,

Popovych²,

Agrawal³

et al. 2021

Preprint

View full text Add to dashboard Cite

Robots can effectively grasp and manipulate objects using their 3D models. In this paper, we propose a simple shape representation and a reconstruction method that outperforms state-of-the-art methods in terms of geometric metrics and enables grasp generation with high precision and success. Our reconstruction method models the object geometry as a pair of depth images, composing the "shell" of the object. This representation allows using image-to-image residual ConvNet architectures for 3D reconstruction, generates object reconstruction directly in the camera frame, and generalizes well to novel object types. Moreover, an object shell can be converted into an object mesh in a fraction of a second, providing time and memory efficient alternative to voxel or implicit representations. We explore the application of shell representation for grasp planning. With rigorous experimental validation, both in simulation and on a real setup, we show that shell reconstruction encapsulates sufficient geometric information to generate precise grasps and the associated grasp quality with over 90% accuracy. Diverse grasps computed on shell reconstructions allow the robot to select and execute grasps in cluttered scenes with more than 93% success rate.

show abstract

“…It has the advantage of being more detail preserving than that of directly regressing the 3-D shapes. For example, the recent work of front2back [47] predicts depth map from an input image, and predict the invisible sides with symmetry. However, their output is point cloud, which requires additional non-trivial post-processing steps [2,12] to deliver the final 3-D mesh.…”

Section: Single Image Based Object Reconstructionmentioning

confidence: 99%

Object Wake-up: 3D Object Rigging from a Single Image

Yang¹,

Zuo²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Given a picture of a chair, could we extract the 3-D shape of the chair, animate its plausible articulations and motions, and render in-situ in its original image space? The above question prompts us to devise an automated approach to extract and manipulate articulated objects in single images. Comparing with previous efforts on object manipulation, our work goes beyond 2-D manipulation and focuses on articulable objects, thus introduces greater flexibility for possible object deformations. The pipeline of our approach starts by reconstructing and refining a 3-D mesh representation of the object of interest from an input image; its control joints are predicted by exploiting the semantic part segmentation information; the obtained object 3-D mesh is then rigged & animated by non-rigid deformation, and rendered to perform in-situ motions in its original image space. Quantitative evaluations are carried out on 3-D reconstruction from single images, an established task that is related to our pipeline, where our results surpass those of the SOTAs by a noticeable margin. Extensive visual results also demonstrate the applicability of our approach.

show abstract

Front2Back: Single View 3D Shape Reconstruction via Front to Back Prediction

Cited by 39 publications

References 31 publications

Sym3DNet: Symmetric 3D Prior Network for Single-View 3D Reconstruction

Sym3DNet: Symmetric 3D Prior Network for Single-View 3D Reconstruction

Object Shell Reconstruction: Camera-centric Object Representation for Robotic Grasping

Object Wake-up: 3D Object Rigging from a Single Image

Contact Info

Product

Resources

About