Efficient Semantic Scene Completion Network with Spatial Group Convolution

Abstract

We introduce Spatial Group Convolution (SGC) for accelerating the computation of 3D dense prediction tasks. SGC is orthogonal to group convolution, which works on spatial dimensions rather than feature channel dimension. It divides input voxels into different groups, then conducts 3D sparse convolution on these separated groups. As only valid voxels are considered when performing convolution, computation can be significantly reduced with a slight loss of accuracy. The proposed operations are validated on semantic scene completion task, which aims to predict a complete 3D volume with semantic labels from a single depth image. With SGC, we further present an efficient 3D sparse convolutional network, which harnesses a multiscale architecture and a coarse-to-fine prediction strategy. Evaluations are conducted on the SUNCG dataset, achieving state-of-the-art performance and fast speed. Code is available at https://github.com/zjhthu/SGC-Release.git

RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion

“…Spatial Group Convolution To improve the computing efficiency of the 3D network. EsscNet [42] is introduced, rather than to conduct the group convolution on feature channel dimension, which adopts the group convolution on the spatial aspect. The drawback of spatial group convolution is that it splits the features manually into separate parts, which cause the performance drops.…”

“…However, the performance of both scene completion and semantic scene completion is around 6% higher than that of SSCNet. Compared with the Essc-Net [42], depth solely is used as the input for a fair comparison, our method is computationally cheaper than EsscNet with 6% reduction in FLOPS and increased performance. For SC and SSC tasks, EsscNet reaches the accuracies of 56.2% (SC) and 26.7% (SSC), and we achieve 59.0% (SC) and 28.9% (SSC).…”

Depth Based Semantic Scene Completion With Position Importance Aware Loss

“…Recently several methods have been proposed for SSC using deep learning techniques [3], [12], [13], [8]. Among them, the most representative work is the SSCNet [3] which conducts the semantic labeling and scene completion simultaneously and also proves that these two tasks can benefit from each other.…”

“…Although better results have been achieved compared with the previous methods, SSCNet ignores the finegrained information of depth. Zhang et al [13] introduces spatial group convolution (SGC) to reduce the computation costs but with poor performance than SSCNet [3]. SEGCloud [9] employs fine-grained 3D point as input but the computing and memory costs are incredibly high.…”

3D Semantic Scene Completion from a Single Depth Image Using Adversarial Training

“…On NYU Kinect, the proposed approaches perform worse than the baseline [5] for semantic scene completion, but better for scene completion. The only approach that fairly outperforms the baseline is [23]. All the other approaches use either an additional modality (RGB images) [8], pretrain on SUNCG [5], or do both [9].…”