Multi-Path Region Mining for Weakly Supervised 3D Semantic Segmentation on Point Clouds

Abstract

Semantic segmentation on 3D point clouds is an important task for 3D scene understanding. While dense labeling on 3D data is expensive and time-consuming, only a few works address weakly supervised semantic point cloud segmentation methods to relieve the labeling cost by learning from simpler and cheaper labels. Meanwhile, there are still huge performance gaps between existing weakly supervised methods and state-of-the-art fully supervised methods. In this paper, we train a semantic point cloud segmentation network with only a small portion of points being labeled. We argue that we can better utilize the limited supervision information as we densely propagate the supervision signal from the labeled points to other points within and across the input samples. Specifically, we propose a cross-sample feature reallocating module to transfer similar features and therefore re-route the gradients across two samples with common classes and an intra-sample feature redistribution module to propagate supervision signals on unlabeled points across and within point cloud samples. We conduct extensive experiments on public datasets S3DIS and ScanNet. Our weakly supervised method with only 10% and 1% of labels can produce compatible results with the fully supervised counterpart.

Box2Seg: Learning Semantics of 3D Point Clouds with Box-Level Supervision

“…Finally, the trained model is used to infer the semantic labels of the raw point clouds. Note that, we are aware that the generated pseudo labels are not completely correct, but similar to [48], we found that the final segmentation results are totally acceptable, even trained with these flawed labels.…”

“…That is, learning semantics from other indirect formats of weak supervision or a small fraction of labeled points. MPRM [48] and SegGroup [40] are proposed to generate pseudo pointlevel labels from subcloud-level labels or seg-level labels. SQN [19], Xu's work [55] and SceneContrast [18] are introduced to learn semantic segmentation from a small fraction of randomly annotated or actively labeled points.…”

“…To reduce the annotation cost, a handful of recent methods [55], [19], [18], [40], [48], [35] have been proposed to investigate weakly supervised semantic segmentation of 3D point clouds. These existing approaches can be divided into two categories according to their weak supervision settings: methods with limited point supervision, methods with inexact supervision.…”

“…Instead of using pointlevel supervision, these methods attempt to learn from various inexact supervision, including subcloud-level tags [35], [48], seg-level labels [40], and 2D pixel-wise labels [46]. Wei et al [49] first train a classifier using subcloud-level labels, and then introduce a multi-path region mining module to extract the object localization region cues. They finally generate pseudo labels for the training of their segmentation network.…”

GECNN for Weakly Supervised Semantic Segmentation of 3D Point Clouds

“…The multi-path region mining (MPRM) [6] model leverages class activate map to explore object localization cues from weak labels that indicates categrories of input spherical point samples at inexact sub-cloud level, different attention mechanisms are explored to learn a point cloud scene segmentation network by mining long-range spatial context, channel inter-dependency and global context on raw 3D data. But the limited number of pooling layers will inevitably restrict the size of receptive field thus hard for complex scene abstraction.…”