Learning to Detect 3D Symmetry From Single-View RGB-D Images With Weak Supervision

Abstract

3D symmetry detection is a fundamental problem in computer vision and graphics. Most prior works detect symmetry when the object model is fully known, few studies symmetry detection on objects with partial observation, such as single RGB-D images. Recent work addresses the problem of detecting symmetries from incomplete data with a deep neural network by leveraging the dense and accurate symmetry annotations. However, due to the tedious labeling process, full symmetry annotations are not always practically available. In this work, we present a 3D symmetry detection approach to detect symmetry from single-view RGB-D images without using symmetry supervision. The key idea is to train the network in a weakly-supervised learning manner to complete the shape based on the predicted symmetry such that the completed shape be similar to existing plausible shapes. To achieve this, we first propose a discriminative variational autoencoder to learn the shape prior in order to determine whether a 3D shape is plausible or not. Based on the learned shape prior, a symmetry detection network is present to predict symmetries that produce shapes with high shape plausibility when completed based on those symmetries. Moreover, to facilitate end-to-end network training and multiple symmetry detection, we introduce a new symmetry parametrization for the learning-based symmetry estimation of both reflectional and rotational symmetry. The proposed approach, coupled symmetry detection with shape completion, essentially learns the symmetry-aware shape prior, facilitating more accurate and robust symmetry detection. Experiments demonstrate that the proposed method is capable of detecting reflectional and rotational symmetries accurately, and shows good generality in challenging scenarios, such as objects with heavy occlusion and scanning noise. Moreover, it achieves state-of-the-art performance, improving the F1-score over the existing supervised learning method by 2%-11% on the ShapeNet and ScanNet datasets.

SymmetryGrasp: Symmetry-Aware Antipodal Grasp Detection From Single-View RGB-D Images

“…Prior works of symmetry detection are only applicable to synthetic objects in the virtual world, limiting the effectiveness of robotic applications. Several recent works studies detecting symmetry from partially observed data with various formats, such as RGB image [42], RGB-D image [32], [33], and 3D volume [10]. Our method is inspired by the above methods.…”

Self-Writer: Clusterable Embedding Based Self-Supervised Writer Recognition from Unlabeled Data

“…Another approach is to create a synthetic categorization task where one can create a surrogate class by altering a single image multiple times through translations, color shifts, and rotations [46]. Furthermore, in [47], in order to detect 3D symmetry from single-view RGB-D images, the author uses weak supervision to detect objects.…”

Sensitivity study of multi-field information maps of typical landslides in mining areas based on transfer learning

“…The geographic information system and the advancement of artificial intelligence (AI) technology will promote the development of more efficient and accurate studies on the sensitivity of the landslide inventory, which makes the evaluation of contributing factor system more reasonable (Ghorbanzadeh et al, 2022b). More intelligent models are applied to landslide susceptibility, and all kinds of machine learning methods, including logistic, classification and regression tree (CART), SVM, and transfer learning, have been widely used (Huo et al, 2019;Ghorbanzadeh et al, 2022c;Shi et al, 2022;Wang et al, 2022). C5.0 decision tree, random forest, and support vector machine are used to partition landslide sensitivity and compare its performance in the coal mining area.…”