Extending Convolution to Semantic Segmentation with Graph Neural Networks

Abstract

Scene understanding enables multiple applications such as robotic automation. To enable scene understanding, semantic segmentation is an essential component. Highly accurate per-voxel semantic labels can efficiently be obtained through the use of encoder-decoder Convolutional Neural Networks (CNNs). However, scene understanding requires instances of objects to be discovered in a scene. Per-voxel labelling is unable to distinguish between separate instances of a given object. A novel method is proposed that enables segment-level predictions to be formed from the voxel-level predictions provided by the CNN architecture. This method uses an existing 3D CNN architecture to provide rich and computationally inexpensive features. The scene is then segmented with a graph-based segmentation technique to identify objects in the scene. Finally, a graph-based neural network is used to capture hierarchical information improving overall accuracy. The approach is tested on the S3DIS and ScanNet datasets to provide a comparable baseline to more modern architectures.