Semi-supervised 3D Face Representation Learning from Unconstrained Photo Collections

Learning Spectral Dictionary for Local Representation of Mesh

“…FeaStNet [Verma et al, 2018] proposes a graph-convolution operator that learns a weighting matrix dynamically computed from features. LSA-Conv [Gao et al, 2021] proposed to learn weighting matrices for each vertex of the template to soft-permute the vertex's neighbors so that CNNs can be applied. Our SDConv is a concurrent work with LSA-Conv to construct weighting matrices for each vertex.…”

“…An efficient and effective convolutional operation is desirable and crucial for representation learning of meshes in many 3D tasks, e.g., reconstruction [Tran and Liu, 2018;Gao et al, 2020b], shape correspondence [Groueix et al, 2018], shape synthesis and modeling [Cao et al, 2014b], face recognition [Liu et al, 2018a] and shape segmentation [Ðonlić et al, 2017;Kalogerakis et al, 2010], and graphics applications such as virtual avatar [Cao et al, 2014a]. The success of convolutional neural networks (CNN) in the fields where underlying data are Euclidean structured (e.g., images, audio, computed tomography images) has inspired many researchers * Contact Author Figure 1: Quantitative evaluation of our SDConv+ against LSA-Conv and LSA-small [Gao et al, 2021], Spiral [Bouritsas et al, 2019], COMA [Ranjan et al, 2018], andFeaStNet [Verma et al, 2018] on the DFAUST dataset for latent size d = 32 with the same network architecture, in terms of reconstruction errors, inference time complexity, and parameter size. SDConv+ outperforms LSA-Conv easily by increasing the channel sizes of the network while maintaining a much smaller model size (denoted as SDConv+L).…”

“…However, a spiral order of local neighbors cannot resolve the inconsistency between different vertices. Experiments in Table 2 of the recent work [Gao et al, 2021] show that a randomized order of neighbors can even achieve better results.…”

“…In line with LSA-Conv [Gao et al, 2021], COMA [Ranjan et al, 2018], SpiralNet [Bouritsas et al, 2019], andSpiralNet++ [Gong et al, 2019], SDConv is also designed for fixed-topology meshes. We evaluate our approach on two fundamental testbeds: autoencoder-based reconstruction and 3D shape correspondence.…”

“…In contrast to global methods like COMA [Ranjan et al, 2018] with isotropic filters or predefined local coordinates used in SpiralNet [Bouritsas et al, 2019] using shared model across all the vertices, SDConv personalizes the vertex representation model to enjoy higher local flexibility. Yet compared with the concurrent local structure-aware learning method [Gao et al, 2021] that learns all the weighting matrices for each vertex separately, SDConv notably reduces the model parameter size by a more compact dictionary-based encoding scheme (see Figure 1 for comparison).…”