Semantic and Context Information Fusion Network for View-Based 3D Model Classification and Retrieval

Abstract

In recent years, with the rapid development of 3D technology, view-based methods have shown excellent performance in both 3D model classification and retrieval tasks. In view-based methods, how to aggregate multi-view features is a key issue. There are two commonly used solutions in the existing methods: 1) Use pooling strategy to merge multi-view features, but it ignores the context information contained in the continuous view sequence. 2) Leverage grouping strategy or long short term memory networks (LSTM) to select representative views of the 3D model, however, it easily neglects the semantic information of individual views. In this paper, we propose a novel Semantic and Context information Fusion Network (SCFN) to compensate for these drawbacks. First, we render views from multiple perspectives of the 3D model and extract the raw feature of the individual view by 2D convolutional neural networks (CNN). Then we design the channel attention mechanism (CAM) to exploit the view-wise semantic information. By modeling the correlation among view feature channels, we can assign higher weights to useful feature attributes, while suppressing the useless. Next, we propose a context information fusion module (CFM) to fuse multiple view features to obtain a compact 3D representation. Extensive experiments are conducted on three popular datasets, i.e., ModelNet10, ModelNet40, and ShapeNetCore55, which can demonstrate the superiority of the proposed method comparing to the state-of-the-arts on both 3D classification and retrieval tasks.

3D Modeling of a Virtual Built Environment Using Digital Tools: Kilburun Fortress Case Study

“…According to the up-to-date experience of other studies [11,12], it is possible to combine the proposed approach with methods based on photogrammetry, photo scanning, and laser scanning [10]. This allows for the development of a more detailed model of the built environment and the creation of geodetic measurements in the relevant programs.…”

3D Model Classification Based on Bayesian Classifier with AdaBoost

“…Chen et al [15] propose multimodal support vector machine to combine sift descriptor, outline Fourier transform descriptor, and Zernike moments descriptor to discriminate multiple classes of object, which considers the independence of each modality and the interrelation between them. Liu et al [16] design a semantic and context information fusion network, which extracts contextual information in continuous view sequence and semantic information of individual view. Kim et al [17] give a viewpoint and image-resolution estimation method for view-based 3D shape retrieval from point cloud queries.…”