Semantic Consistency Guided Instance Feature Alignment for 2D Image-Based 3D Shape Retrieval

Zhou, Heyu; Nie, Weizhi; Song, Dan; Hu, Nian; Li, Xuanya; Liu, An-An

doi:10.1145/3394171.3413631

Cited by 21 publications

(14 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…31 and (2) deep cross-domain learning methods including JAN, 32 RevGrad, 33 DLEA, 10 and SC-IFA. 11 For the traditional cross-domain alignment methods, CORAL 29 utilizes the second-order statistics of both source and target data to reduce the domain gap; MEDA 30 aims to train a domain-invariant classifier to achieve the domain alignment; JGSA 31 proposes to solve the problem of negative transfer by aligning the projections of source and target domain. For the deep cross-domain learning methods, JAN 32 align the source and target data distributions at multiple layers in the deep neural network; RevGrad 33 performs gradient reversal to learn domain-invariant features during back-propagation; DLEA 10 proposes a dual-level embedding alignment network.…”

Section: Comparison With State-of-the-artsmentioning

confidence: 99%

“…For the deep cross-domain learning methods, JAN 32 align the source and target data distributions at multiple layers in the deep neural network; RevGrad 33 performs gradient reversal to learn domain-invariant features during back-propagation; DLEA 10 proposes a dual-level embedding alignment network. SC-IFA 11 preserves cross-domain consistency via feature translation from image domain to 3D model domain and vice versa, and reduces the domain gap in an adversarial way.…”

Section: Comparison With State-of-the-artsmentioning

confidence: 99%

“…Visual comparison of the retrieval results between the proposed method and SC-IFA on MI3DOR-2 dataset, where the results of SC-IFA are cited from. 11 The wrong results are marked with a red box.…”

Section: F I G U R Ementioning

confidence: 99%

“…Image-based 3D model retrieval methods [10][11][12] pay attention to narrowing the modal gap and aligning image features and 3D model features. For instance, DLEA 10 proposes a dual-level embedding alignment network.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Source‐enhanced prototypical alignment for single image 3D model retrieval

Song

Wang

Zhang

et al. 2022

Computer Animation & Virtual

Self Cite

View full text Add to dashboard Cite

Single image 3D model retrieval has attracted a lot of attentions with the convenience of organizing large-scale unlabeled 3D models. Existing methods transfer the knowledge from well-annotated 2D images (i.e., source domain) to unlabeled 3D models (i.e., target domain) to improve the discriminability of 3D models and align the feature distributions of 2D images and 3D models. However, during the alignment, the feature learning target of improving the discriminability of 3D models sometimes confuses the boundaries between 2D image categories, where prior methods ignore keeping the discriminability of 2D images. Motivated by this observation, we propose a source-enhanced prototypical alignment framework to first remain the discriminability of 2D images and then guide the category-level cross-domain alignment with better image representations. Specifically, a novel separation and compactness loss is proposed for images to separate the samples from different categories and compact the samples within the same category. Then we perform prototypical alignment to make 2D image features assist in the discriminative feature learning for 3D models. We evaluate the proposed method on the commonly used cross-domain 3D model retrieval benchmarks, namely MI3DOR and MI3DOR-2, and the results demonstrate the effectiveness of the proposed method.

show abstract

Section: Comparison With State-of-the-artsmentioning

confidence: 99%

Section: Comparison With State-of-the-artsmentioning

confidence: 99%

Section: F I G U R Ementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Source‐enhanced prototypical alignment for single image 3D model retrieval

Song

Wang

Zhang

et al. 2022

Computer Animation & Virtual

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, with the tremendous advances of deep learning in recent years [5,8,14,16,21,22,28,32,33], various deep networks have been employed for learning 3D model representation. According to different methods of feature representation, related methods can be roughly divided into two categories: 1) model-based methods [23,26] and 2) view-based methods [5,14,16,24,28,35,36]. In the model-based methods, the feature is directly extracted from the 3D model, and view-based methods place multiple virtual cameras around the 3D model to generate 2D images, which are used as the input data in view-based methods.…”

Section: Introductionmentioning

confidence: 99%

A Novel Patch Convolutional Neural Network for View-based 3D Model Retrieval

Gao

Shao

Guan

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Recently, many view-based 3D model retrieval methods have been proposed and have achieved state-of-the-art performance. Most of these methods focus on extracting more discriminative viewlevel features and effectively aggregating the multi-view images of a 3D model, but the latent relationship among these multi-view images is not fully explored. Thus, we tackle this problem from the perspective of exploiting the relationships between patch features to capture long-range associations among multi-view images. To capture associations among views, in this work, we propose a novel patch convolutional neural network (PCNN) for view-based 3D model retrieval. Specifically, we first employ a CNN to extract patch features of each view image separately. Secondly, a novel neural network module named PatchConv is designed to exploit intrinsic relationships between neighboring patches in the feature space to capture long-range associations among multi-view images. Then, an adaptive weighted view layer is further embedded into PCNN to automatically assign a weight to each view according to the similarity between each view feature and the view-pooling feature. Finally, a discrimination loss function is employed to extract the discriminative 3D model feature, which consists of softmax loss values generated by the fusion classifier and the specific classifier. Extensive experimental results on two public 3D model retrieval benchmarks, namely, the ModelNet40, and ModelNet10, demonstrate that our proposed PCNN can outperform state-of-the-art approaches, with mAP values of 93.67%, and 96.23%, respectively.

show abstract