Mesh Saliency via Weakly Supervised Classification-for-Saliency CNN

Abstract

Recently, effort has been made to apply deep learning to the detection of mesh saliency. However, one major barrier is to collect a large amount of vertex-level annotation as saliency ground truth for training the neural networks. Quite a few pilot studies showed that this task is difficult. In this work, we solve this problem by developing a novel network trained in a weakly supervised manner. The training is end-to-end and does not require any saliency ground truth but only the class membership of meshes. Our Classification-for-Saliency CNN (CfS-CNN) employs a multi-view setup and contains a newly designed two-channel structure which integrates view-based features of both classification and saliency. It essentially transfers knowledge from 3D object classification to mesh saliency. Our approach significantly outperforms the existing state-of-the-art methods according to extensive experimental results. Also, the CfS-CNN can be directly used for scene saliency. We showcase two novel applications based on scene saliency to demonstrate its utility.

Robust and Fast 3-D Saliency Mapping for Industrial Modeling Applications

“…θ = 5.68 o HD = 0.002054 θ = 9.79 o HD = 0.003241 (f) (g) Fig. 9: Heatmaps of saliency mapping and denoising results using the methods: (a) curvature co-occurrence histogram [9], (b) entropy based salient model [11], (c) a mesh saliency [28], (d) mesh saliency via spectral processing [12], (e) Point-wise saliency detection [29], (f) mesh saliency via CNN [30], (g) our approach.…”

“…(a) Original model, and heatmaps visualization of saliency mapping based on: (b) the eigenvalues of small patches (spectral analysis), as described in paragraph IV-B, (c) the RPCA approach (geometrical analysis), as described in paragraph IV-A, (d) Wei et al[9], (e) Tao et al[11], (f) Lee et al[28], (g) Song et al[12], (h) Guo et al[29], (i) Song et al (CNN)[30], (j) our approach.…”

“…Simplification of 3D models using the saliency mapping of different methods, namely: (I) Wei et al[9], (II) Tao et al[11], (III) Lee et al[28], (IV) Song et al[12], (V) Guo et al[29], (VI) Song et al (CNN)[30] and (VII) our approach respectively (from up to down). (a) Heatmap visualization of the saliency mapping and simplified results using different simplification approaches, Cad(19,398 points): (b) 2000 (∼ 10.3%), (c) 4000 (∼ 20.6%), Block (8,771 points): (b) 2000 (∼ 22.8%), (c) 4000 (∼ 45.6%), Part Lp (4,261points): (b) 500 (∼ 11.7%), (c) 1000 (∼ 23.4%), Coverrear Lp (7,872 points): (b) 2000 (∼ 25.4%), (c) 3000 (∼ 38.1%).…”

Learning on 3D Meshes With Laplacian Encoding and Pooling

“…Recently, Song et al [21] use a multi-view representation of meshes and apply a CNN on it. Some other works design regular CNNs on surface meshes by parametrizing the manifold.…”

Mesh Saliency Detection Using Convolutional Neural Networks

“…Convolutional neural networks(CNNs) have been also extensively used for the extraction of saliency maps in image processing [14]. A recent study employs CNNs to extract saliency maps on 3D meshes utilizing a multi-view setup [15]. Motivated by the aforementioned challenges and open issues this work employs CNNs to automatically derive saliency maps from 3D geometries facilitating the processing of very large and dense meshes with lower complexity while enabling parallelization.…”