Three‐dimensional shape reconstruction of objects from a single depth view using deep U‐Net convolutional neural network with bottle‐neck skip connections

Abstract

Three‐dimensional (3D) shape reconstruction of objects requires multiple scans and complex reconstruction algorithms. An alternative approach is to infer the 3D shape of an object from a single depth image (i.e. single depth view). This study presents such a 3D shape reconstructor based on U‐Net 3D‐convolutional neural network (3D‐CNN) with bottle‐neck skipped connection blocks (U‐Net BNSC 3D‐CNN) to infer the 3D shapes of objects from only a single depth view. The BNSC block is a fully convolutional block that uses skip connections to improve the performance of the sequential 3D‐convolutional layers of U‐Net. The primary advantage of U‐Net BNSC 3D‐CNN is improving the accuracy of shape reconstruction while reducing the computational load. The evaluation of the proposed U‐Net BNSC 3D‐CNN uses unseen views from trained and untrained objects with two public databases, ShapeNet and Grasp database. Our reconstructor achieves 72.17% and 69.97% accuracy in terms of the Jaccard similarity index for trained and untrained objects, respectively, with the ShapeNet database, whereas previous reconstructor based on 3D‐CNN achieves 66.43% and 58.35%. With Grasp database, our reconstructor achieves 87.03% and 85.35%, whereas 3D‐CNN 76.52% and 76.02%. Also, our U‐Net BNSC 3D‐CNN reduces the computational load of the standard 3D‐CNN reconstructor by 6.67% in the computation time and by 98.69% in the number of trainable parameters.

Visual reconstruction method of architectural space under laser point cloud big data

“…Statistically, this method is used to visually reconstruct the building space, and compared with the 3D shape reconstruction method of view object based on deep u-net convolutional neural network in literature [11], the reconstruction accuracy, reconstruction completion and reconstruction smoothness of the two methods reconstruction results are shown in Fig. 4.…”

“…This method applied the distributed computing method to large-scale point cloud data processing, and introduced the solution of B-EagleV to realize the visualization of buildings in civil engineering, which provided a foundation for data management, progress monitoring and other applications in the construction process. Anazco et al used the deep convolution neural network to reconstruct the threedimensional view [11]. This method set up a three-dimensional shape reconstructor, and used full convolution blocks to improve the accuracy of material reconstruction and reduce the calculation of three-dimensional reconstruction.…”

Deep Learning-Based Ensemble Approach for Autonomous Object Manipulation with an Anthropomorphic Soft Robot Hand

“…The 3D convolutional layers are used because of their larger 3D receptive field [35]. The skipped connections propagate the features and training error to the whole network, and the bottle-neck layers reduce the trainable parameters [22].…”

“…Instead of matching or using multiple images, 3D object shapes could be reconstructed using DL. For instance, Valarezo et al [22] proposed a 3D object shape reconstructor based on U-Net 3D-CNN with a bottle-neck skip connection block (3D U-Net-based BNSC) to reconstruct trained and untrained objects from a single partial view. The idea is to reconstruct the 3D shapes of objects using 3D U-Net-based BNSC to generate the grasping position and orientation angle.…”

Deep Convolution Neural Network in Remote Sensing Monitoring of Water Source Pollution Sources

“…Some researchers use open source PostgresSQL, MapGuide and Ppaper to combine remote sensing and GIS technology and create a seabed water quality management system on WebGIS to support the decision-making of competent authorities or society on the management of seabed pollution [2]. Some scholars believe that it is necessary to continuously monitor the water quality, systematically assess the water quality of the basin, understand the overall water pollution situation of the basin, determine the main pollution factors and levels of water pollution, and explain the temporal and spatial distribution of water pollution characteristics in order to comprehensively control the water pollution in river pools [3][4].…”