VConv-DAE: Deep Volumetric Shape Learning Without Object Labels

Abstract

With the advent of affordable depth sensors, 3D capture becomes more and more ubiquitous and already has made its way into commercial products. Yet, capturing the geometry or complete shapes of everyday objects using scanning devices (e.g. Kinect) still comes with several challenges that result in noise or even incomplete shapes. Recent success in deep learning has shown how to learn complex shape distributions in a data-driven way from large scale 3D CAD Model collections and to utilize them for 3D processing on volumetric representations and thereby circumventing problems of topology and tessellation. Prior work has shown encouraging results on problems ranging from shape completion to recognition. We provide an analysis of such approaches and discover that training as well as the resulting representation are strongly and unnecessarily tied to the notion of object labels. Thus, we propose a full convolutional volumetric auto encoder that learns volumetric representation from noisy data by estimating the voxel occupancy grids. The proposed method outperforms prior work on challenging tasks like denoising and shape completion. We also show that the obtained deep embedding gives competitive performance when used for classification and promising results for shape interpolation.

High Level 3D Structure Extraction from a Single Image Using a CNN-Based Approach

“…The volumetric representation is processed by 3D ShapeNets to identify the observed shape, the free space and the occluded space. The method presented by [31] proposes a network for deep volumetric shape learning. Given a collection of shapes of various objects and their different poses, the network learns the distributions of shapes of various classes by predicting the missing sections.…”

“…Although these methods have shown promising results on 3D shapes’ extraction, in most cases, they are limited to specific objects. Furthermore, these methods do not present polyhedral structures’ extraction on buildings or outdoor scenes [31,32,33]. On the other hand, the approaches that have promising results in outdoor scenes, in most cases, are those limited to plane sections’ extraction without providing polyhedral structures’ extraction [15,29].…”

Trilateral convolutional neural network for 3D shape reconstruction of objects from a single depth view

“…The decoder uses two deconvolution layers: the filter size of the first layers and feature map were set to , with a stride of two, and the second layer has with a stride of three. This modified network was named AE‐CNN, according to the autoencoder introduced in [15].…”

“…In addition, autoencoder‐based deep learning approaches have been used to enhance the performance of 3D shape reconstruction. Sharma et al [15] developed an end‐to‐end reconstruction technique, known as a fully volumetric convolutional denoising autoencoder (VConv‐DAE). They used convolution layers to obtain a latent representation of the input object and a learnable upsampling convolution filter (i.e.…”

“…In this paper, the proposed Tri‐CNN is compared against four deep learning approaches presented by Varley et al [3, 4] [i.e. fully connected convolutional neural network (F‐CNN)], Sharma et al [15] [i.e. autoencoder CNN (AE‐CNN)], Yang et al [21] [i.e.…”