Multi-Level 3D CNN for Learning Multi-Scale Spatial Features

Ghadai, Sambit; Lee, Xian Yeow; Balu, Aditya; Sarkar, Soumik; Krishnamurthy, Adarsh

doi:10.1109/cvprw.2019.00150

Cited by 18 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Other works also use hierarchical representations like octrees [23] to further lessen the effect of this issue. Related to this, Ghadai et al [24] propose a multi-resolution CNN for object recognition based on a multi-level voxel grid. In addition to this, with upcoming hardware and more available memory, it is expected that in the near future more detailed voxelizations could be used with CNNs, so an efficient voxelization pipeline as the one proposed here would be very useful.…”

Section: Related Workmentioning

confidence: 99%

A GPU-Based Framework for Generating Implicit Datasets of Voxelized Polygonal Models for the Training of 3D Convolutional Neural Networks

et al. 2020

View full text Add to dashboard Cite

In this paper we present an efficient GPU-based framework to dynamically perform the voxelization of polygonal models for training 3D convolutional neural networks. It is designed to manage the dataset augmentation by using efficient geometric transformations and random vertex displacements in GPU. With the proposed system, every voxelization is carried out on-the-fly for directly feeding the network. The computing performance with this approach is much better than with the standard method, that carries out every voxelization of each model separately and has much higher data processing overhead. The core voxelization algorithm works by using the standard rendering pipeline of the GPU. It generates binary voxels for both the interior and the surface of the models. Moreover, it is simple, concise and very compatible with commodity hardware, as it neither uses complex data structures nor needs vendor-specific hardware or additional dependencies. This framework dramatically reduces the input/output operations, and completely eliminates the storage footprint of the voxelization dataset, managing it as an implicit dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

A GPU-Based Framework for Generating Implicit Datasets of Voxelized Polygonal Models for the Training of 3D Convolutional Neural Networks

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Model-based methods [39,41,49] can capture the 3D structure of original shape, such as polygon mesh [39], voxel grid [12] and point cloud [38]. For example, Ghadai et al [12] proposed an approach that used a multi-level unstructured voxel representation of spatial data to learn features. The view-based 3D object retrieval methods employ a set of rendered images to represent 3D objects [8].…”

Section: Related Work 21 3d Object Retrievalmentioning

confidence: 99%

Domain-Specific Alignment Network for Multi-Domain Image-Based 3D Object Retrieval

Song

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

2D image-based 3D object retrieval is a very important task in computer vision and big data management. Conventional image-based 3D object retrieval usually assumes that the images are from one single domain. However, for real applications, 2D images may be from multiple domains (e.g., real image, sketch, and quick draw). It raises significant challenges for this task since these 2D images have a great domain gap with each other as well as a great modality gap with 3D objects. To address these issues, we propose an unsupervised Domain-Specific Alignment Network (DSAN) for multidomain image-based 3D object retrieval. The proposed method aims to reduce domain discrepancy by domain-specific alignment network with multi-level moment matching, including first-order moment and second-order moment. Based on the observation that for any given sample, different domain classifiers should output the same label, we design a domain-specific classifier alignment module. To our knowledge, the proposed method is the first unsupervised work to align multiple-domain 2D images with 3D objects in an end-to-end manner. The multi-domain dataset MDI3D is utilized to advocate the research on this task, and the extensive experimental results demonstrate the superiority of the proposed method. CCS CONCEPTS • Computing methodologies → Computer vision; Visual contentbased indexing and retrieval; • Information systems → Multimedia and multimodal retrieval.

show abstract

“…Model-based 3D model retrieval methods [5][6][7] put emphasis on learning discriminative 3D features with certain 3D forms, such as point cloud, 5 voxel, 8 and multi-view 9 representations. For example, PVRNet 6 well fuses the multi-view features and point cloud features based on a relation score module to obtain a unified representation for 3D shapes.…”

Section: Introductionmentioning

confidence: 99%

“…Ghadai et al 8 propose a multi-level voxel grid to obtain multi-level voxel representations of 3D models. However, compared with 2D images, 3D models are not widely available at hand and also expensive to annotate.…”

Section: Introductionmentioning

confidence: 99%

Source‐enhanced prototypical alignment for single image 3D model retrieval

Song

Wang

Zhang

et al. 2022

Computer Animation & Virtual

View full text Add to dashboard Cite

Single image 3D model retrieval has attracted a lot of attentions with the convenience of organizing large-scale unlabeled 3D models. Existing methods transfer the knowledge from well-annotated 2D images (i.e., source domain) to unlabeled 3D models (i.e., target domain) to improve the discriminability of 3D models and align the feature distributions of 2D images and 3D models. However, during the alignment, the feature learning target of improving the discriminability of 3D models sometimes confuses the boundaries between 2D image categories, where prior methods ignore keeping the discriminability of 2D images. Motivated by this observation, we propose a source-enhanced prototypical alignment framework to first remain the discriminability of 2D images and then guide the category-level cross-domain alignment with better image representations. Specifically, a novel separation and compactness loss is proposed for images to separate the samples from different categories and compact the samples within the same category. Then we perform prototypical alignment to make 2D image features assist in the discriminative feature learning for 3D models. We evaluate the proposed method on the commonly used cross-domain 3D model retrieval benchmarks, namely MI3DOR and MI3DOR-2, and the results demonstrate the effectiveness of the proposed method.

show abstract

Multi-Level 3D CNN for Learning Multi-Scale Spatial Features

Cited by 18 publications

References 11 publications

A GPU-Based Framework for Generating Implicit Datasets of Voxelized Polygonal Models for the Training of 3D Convolutional Neural Networks

A GPU-Based Framework for Generating Implicit Datasets of Voxelized Polygonal Models for the Training of 3D Convolutional Neural Networks

Domain-Specific Alignment Network for Multi-Domain Image-Based 3D Object Retrieval

Source‐enhanced prototypical alignment for single image 3D model retrieval

Contact Info

Product

Resources

About