Unsupervised Multi-View CNN for Salient View Selection and 3D Interest Point Detection

Song, Ran; Zhang, Wei; Zhao, Yitian; Liu, Yonghuai

doi:10.1007/s11263-022-01592-x

Cited by 16 publications

(11 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Apart from measures covered in [5] there are deep learning approaches [8,9,10], which usually aim to select views based on saliency and human perception.…”

Section: Related Workmentioning

confidence: 99%

Viewpoint Selection for Texture Reconstruction with Inverse Rendering

Sanzharov,

Frolov

2023

Proceedings of the 33rd International Conference on Computer Graphics and Vision

View full text Add to dashboard Cite

Viewpoint selection methods have a variety of applications in different fields of computer graphics and computer vision, including shape retrieval, scientific visualization, image-based modeling and others. In this paper we investigate the applicability of existing viewpoint selection methods to the problem of textures reconstruction using inverse rendering. First, we use forward rendering to produce path-traced images of a textured object. Then we apply different view quality metrics to select a set of images for texture reconstruction. Finally, we perform material and texture reconstruction using these image sets and evaluate the quality of the results. We show that using viewpoint selection methods allows to achieve faster inverse rendering times while maintaining quality of the results.

show abstract

“…Apart from measures covered in [5] there are deep learning approaches [8,9,10], which usually aim to select views based on saliency and human perception.…”

Section: Related Workmentioning

confidence: 99%

Viewpoint Selection for Texture Reconstruction with Inverse Rendering

Sanzharov,

Frolov

2023

Proceedings of the 33rd International Conference on Computer Graphics and Vision

View full text Add to dashboard Cite

show abstract

“…Scene text detection has garnered significant attention in computer vision due to its fundamental role as the initial step for end-to-end text recognition Du et al 2022;Zheng et al 2023a). Driven by the rapid development of deep learning, scene text detection methods have made significant progress (Song et al 2022;Wang et al 2022b;Fang et al 2022;Song et al 2022;Chen et al 2022;Song et al 2023;Qin et al 2023;Zheng et al 2023b;Du et al 2023;Wang et al 2023;Shu et al 2023).…”

Section: Introductionmentioning

confidence: 99%

LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network

Su,

Chen,

Shao

et al. 2024

AAAI

View full text Add to dashboard Cite

Recently, regression-based methods, which predict parameterized text shapes for text localization, have gained popularity in scene text detection. However, the existing parameterized text shape methods still have limitations in modeling arbitrary-shaped texts due to ignoring the utilization of text-specific shape information. Moreover, the time consumption of the entire pipeline has been largely overlooked, leading to a suboptimal overall inference speed. To address these issues, we first propose a novel parameterized text shape method based on low-rank approximation. Unlike other shape representation methods that employ data-irrelevant parameterization, our approach utilizes singular value decomposition and reconstructs the text shape using a few eigenvectors learned from labeled text contours. By exploring the shape correlation among different text contours, our method achieves consistency, compactness, simplicity, and robustness in shape representation. Next, we propose a dual assignment scheme for speed acceleration. It adopts a sparse assignment branch to accelerate the inference speed, and meanwhile, provides ample supervised signals for training through a dense assignment branch. Building upon these designs, we implement an accurate and efficient arbitrary-shaped text detector named LRANet. Extensive experiments are conducted on several challenging benchmarks, demonstrating the superior accuracy and efficiency of LRANet compared to state-of-the-art methods. Code is available at: https://github.com/ychensu/LRANet.git

show abstract

“…Early research on this issue relied on manually defined rules [6] or shallow features [7], which are often data-specific [8], have limited generalization, and depend on strong priors like object orientation [9]. Recently, multi-view-based viewpoint selection approaches overcome these limitations by improving the ability to generalize to different categories [10], providing a deeper understanding of human viewpoint preferences [11], and reducing the reliance on prior knowledge [1], thereby achieving more robust and effective viewpoint predictions.…”

Section: Introductionmentioning

confidence: 99%

Fast Best Viewpoint Selection with Geometry-Enhanced Multiple Views and Cross-Modal Distillation

Cao,

Han,

Yang

et al. 2024

Preprint

View full text Add to dashboard Cite

Despite significant progress on the mesh-based Best Viewpoint Selection (BVS) problem using multi-views, the current state-of-the-art BVS method requires 20-30 rendering views and is limited to selecting from a predefined sequence of viewpoint samples, which may miss optimal viewpoints and precludes its use in situations where response time is critical. To address the limitations, we present a new dual-branch fast best viewpoint selection regression model that significantly reduces reliance on a large number of input views, enables continuous perspective prediction, and enhances interactive response speeds. Our approach incorporates a geometry-enhanced multi-view feature extractor paired with a learnable token and utilizes a cross-modal distillation method to enrich the model’s understanding of 3D structures. Specifically, by embedding view features into a dimensionally matched learnable token and processing it through three cascaded self-attention layers, the resulting token effectively encapsulates fused features that are better suited to the viewpoint selection task.In addition, to reduce the number of views required, we incorporate cross-modal distillation into the BVS solution by imposing alignment constraints between 3D geometry descriptors and fused multi-view expressions, effectively avoiding the computational requirements of dozens of rendering views. Our experimental results on public benchmarks show that our method is approximately 35 times faster than the SOTA method when only six views are adopted, while also achieving the best quantitative metrics.

show abstract

Unsupervised Multi-View CNN for Salient View Selection and 3D Interest Point Detection

Cited by 16 publications

References 54 publications

Viewpoint Selection for Texture Reconstruction with Inverse Rendering

Viewpoint Selection for Texture Reconstruction with Inverse Rendering

LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network

Fast Best Viewpoint Selection with Geometry-Enhanced Multiple Views and Cross-Modal Distillation

Contact Info

Product

Resources

About