Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly

Abstract

On robotics computer vision tasks, generating and annotating large amounts of data from real-world for the use of deep learning-based approaches is often difficult or even impossible. A common strategy for solving this problem is to apply simulation-to-reality (sim2real) approaches with the help of simulated scenes. While the majority of current robotics vision sim2real work focuses on image data, we present an industrial application case that uses sim2real transfer learning for point cloud data. We provide insights on how to generate and process synthetic point cloud data in order to achieve better performance when the learned model is transferred to real-world data. The issue of imbalanced learning is investigated using multiple strategies. A novel patch-based attention network is proposed additionally to tackle this problem.

Advanced Feature Learning on Point Clouds Using Multi-Resolution Features and Learnable Pooling

“…As shown in Table 1, modern feature learning neural networks for point cloud data have been conducted on classification tasks using the ModelNet40 and ScanObjectNN datasets, as well as part segmentation tasks using the ShapeNetPart dataset [1,6,7,[11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27]. Most studies originate from the pioneering work on PointNet by Qi et al [1].…”

“…In contrast to point cloud networks, point cloud sampling remains less explored, with random sampling and farthest point sampling being the most common methods. Wu et al [26] propose a non-generative Attention-based Point Cloud Edge Sampling method (APES) that captures salient points in the point cloud outline and demonstrates superior performance on common benchmark tasks. Furthermore, few-shot learning for point clouds has been explored.…”

Unsupervised Dual Transformer Learning for 3-D Textured Surface Segmentation

“…W ITH the widespread use of 3-D cameras and scanning devices to capture the rich geometrical properties of object surfaces, many computer vision-based interdisciplinary applications have emerged in recent years. A large volume of work has been addressing the problem of segmenting, classifying, and retrieving 3-D shapes based on their similarities using triangle mesh and point clouds as input [1], [2], [3], [4], [5], [6], [7]. However, the segmentation and classification of 3-D geometric textures (or, simply, 3-D textures) are less explored.…”