Unsupervised Multi-view CNN for Salient View Selection of 3D Objects and Scenes

Song, Ran; Zhang, Wei; Zhao, Yitian; Liu, Yonghuai

doi:10.1007/978-3-030-58529-7_27

Cited by 4 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But in general, the methods based on low-level attributes perform significantly worse than those based on deep neural networks which potentially learn some high-level attributes of 3D objects. It is also worth mentioning that the number of objects for which the UMVCNN-VGG gave the lowest VSE is 20 as reported in Song et al (2020b) while it is 15 according to Table 1. This is because the updated method with GDA (i.e.…”

Section: Quantitative Resultsmentioning

confidence: 82%

See 1 more Smart Citation

Unsupervised Multi-View CNN for Salient View Selection and 3D Interest Point Detection

et al. 2022

Self Cite

View full text Add to dashboard Cite

We present an unsupervised 3D deep learning framework based on a ubiquitously true proposition named by us view-object consistency as it states that a 3D object and its projected 2D views always belong to the same object class. To validate its effectiveness, we design a multi-view CNN instantiating it for salient view selection and interest point detection of 3D objects, which quintessentially cannot be handled by supervised learning due to the difficulty of collecting sufficient and consistent training data. Our unsupervised multi-view CNN, namely UMVCNN, branches off two channels which encode the knowledge within each 2D view and the 3D object respectively and also exploits both intra-view and inter-view knowledge of the object. It ends with a new loss layer which formulates the view-object consistency by impelling the two channels to generate consistent classification outcomes. The UMVCNN is then integrated with a global distinction adjustment scheme to incorporate global cues into salient view selection. We evaluate our method for salient view section both qualitatively

show abstract

Section: Quantitative Resultsmentioning

confidence: 82%

“…A preliminary version of this work was published as a poster presentation in European Conference on Computer Vision (ECCV'20) (Song et al, 2020b) 1 , which has been used as a baseline (i.e. UMVCNN-VGG in Tables 1 and 2) for comparisons in this paper.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Multi-View CNN for Salient View Selection and 3D Interest Point Detection

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…We utilize a shared ResNet-50 network to extract the multi-view features. As depicted in [31], each point could be projected to a pixel on a certain view. Inspired by this, we obtain the position embedding via the index of the image and the coordinate of the pixel.…”

Section: Multi-view Feature Extractormentioning

confidence: 99%

Fast Best Viewpoint Selection with Geometry-Enhanced Multiple Views and Cross-Modal Distillation

Cao,

Han,

Yang

et al. 2024

Preprint

View full text Add to dashboard Cite

Despite significant progress on the mesh-based Best Viewpoint Selection (BVS) problem using multi-views, the current state-of-the-art BVS method requires 20-30 rendering views and is limited to selecting from a predefined sequence of viewpoint samples, which may miss optimal viewpoints and precludes its use in situations where response time is critical. To address the limitations, we present a new dual-branch fast best viewpoint selection regression model that significantly reduces reliance on a large number of input views, enables continuous perspective prediction, and enhances interactive response speeds. Our approach incorporates a geometry-enhanced multi-view feature extractor paired with a learnable token and utilizes a cross-modal distillation method to enrich the model’s understanding of 3D structures. Specifically, by embedding view features into a dimensionally matched learnable token and processing it through three cascaded self-attention layers, the resulting token effectively encapsulates fused features that are better suited to the viewpoint selection task.In addition, to reduce the number of views required, we incorporate cross-modal distillation into the BVS solution by imposing alignment constraints between 3D geometry descriptors and fused multi-view expressions, effectively avoiding the computational requirements of dozens of rendering views. Our experimental results on public benchmarks show that our method is approximately 35 times faster than the SOTA method when only six views are adopted, while also achieving the best quantitative metrics.

show abstract

Fast best viewpoint selection with geometry-enhanced multiple views and cross-modal distillation

Cao,

Han,

Yang

et al. 2024

Vis Comput

View full text Add to dashboard Cite

Unsupervised Multi-view CNN for Salient View Selection of 3D Objects and Scenes

Cited by 4 publications

References 34 publications

Unsupervised Multi-View CNN for Salient View Selection and 3D Interest Point Detection

Unsupervised Multi-View CNN for Salient View Selection and 3D Interest Point Detection

Fast Best Viewpoint Selection with Geometry-Enhanced Multiple Views and Cross-Modal Distillation

Fast best viewpoint selection with geometry-enhanced multiple views and cross-modal distillation

Contact Info

Product

Resources

About