A Deep Learning Method for 3D Object Classification and Retrieval Using the Global Point Signature Plus and Deep Wide Residual Network

Abstract

A vital and challenging task in computer vision is 3D Object Classification and Retrieval, with many practical applications such as an intelligent robot, autonomous driving, multimedia contents processing and retrieval, and augmented/mixed reality. Various deep learning methods were introduced for solving classification and retrieval problems of 3D objects. Almost all view-based methods use many views to handle spatial loss, although they perform the best among current techniques such as View-based, Voxelization, and Point Cloud methods. Many views make network structure more complicated due to the parallel Convolutional Neural Network (CNN). We propose a novel method that combines a Global Point Signature Plus with a Deep Wide Residual Network, namely GPSP-DWRN, in this paper. Global Point Signature Plus (GPSPlus) is a novel descriptor because it can capture more shape information of the 3D object for a single view. First, an original 3D model was converted into a colored one by applying GPSPlus. Then, a 32 × 32 × 3 matrix stored the obtained 2D projection of this color 3D model. This matrix was the input data of a Deep Residual Network, which used a single CNN structure. We evaluated the GPSP-DWRN for a retrieval task using the Shapnetcore55 dataset, while using two well-known datasets—ModelNet10 and ModelNet40 for a classification task. Based on our experimental results, our framework performed better than the state-of-the-art methods.

Towards the fully automated monitoring of ecological communities

“…For example, the 100 Island Challenge ( https://100islandchallenge.org/ ) associates classical field surveys with innovative imaging and data technologies to reconstruct 100 m 2 coral reefs digitally and in 3D, from which all corals are individually annotated and classified at the species level (Naughton et al, 2015 ). By combining this workflow and the vast amount of labelled 3D coral structures it has generated with approaches aiming at automatically classifying 3D objects such as MeshCNN (Hanocka et al, 2019 ) and Global point Signature Plus & Deep Wide Residual Network (Hoang et al, 2021 ), we could automate coral habitat mapping in the future. Moreover, the development of autonomous underwater vehicles would help achieving automated surveys (Modasshir & Rekleitis, 2020 ; Ordoñez Avila et al, 2021 ), while coupling these surveys with acoustic monitoring would scale up our understanding about how coral habitats promote surrounding biodiversity (Lin et al, 2021 ) (Figure 6a ).…”

Cube of Space Sampling for 3D Model Retrieval

“…Similarly, for the index issue, the 3D spatial index could be considered to speed up the calculation. Moreover, further performance comparisons could be conducted based on some deep-learning techniques, such as convolutional neural networks [30][31][32] and other feature representations, such as view-based 3D model feature representation [33,34]. Finally, it would be interesting to examine the performance of CSS for different types of 3D objects, such as objects composed of multiple parts that are not watertight, and the sensitivity of CSS when applied to arbitrarily rotated models, or swapping the XYZ axes.…”

Robust Mesh Segmentation Using Feature-Aware Region Fusion

“…For example, Wang et al [ 38 ] introduced a similarity group proposal network for instance segmentation on point clouds. Furthermore, the segmentation problem is closely related to many other 3D processing problems, such as object classification [ 39 ], shape unfolding [ 40 ], and surface denoising [ 41 , 42 ].…”