Enhanced facial expression recognition using 3D point sets and geometric deep learning

Nguyen, Duc-Phong; Tho, M.C. Ho Ba; Dao, Tien Tuan

doi:10.1007/s11517-021-02383-1

Cited by 11 publications

(10 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It contains relatively more exterior variations (including head poses) and occlusions (hands, hair, eyeglasses) in the samples. We followed the protocol in [ 3 , 11 , 15 ], which utilizes 65 subjects with 7 expressions denoted as AN (anger), HA (happiness), FE (fear), SA (sadness), SU (surprise), DI (disgust), and NE (neutral).…”

Section: Resultsmentioning

confidence: 99%

“…3D FER methods have succeeded in mining prominent geometrical details by SPD (Symmetric Positive Matrix) [ 15 ], conformal mapping [ 3 ], depth map [ 16 ], and recently with the prevalent statistical 3D Morphable Model (3DMM) [ 17 ] and point sets [ 11 ]. Among these, Refs.…”

Section: Related Workmentioning

confidence: 99%

“…In the geometric learning branch, Ref. [ 15 ] relies on a predefined mesh structure, while [ 11 ] has been the first to adopt a geometric deep learning framework (i.e., PointNet [ 19 ]) with 3D FER. This inspires our motivation to explore the above approach in order to realize further improvements in performance.…”

Section: Related Workmentioning

confidence: 99%

“…In a prior trial in 3D FER with GDL, Ref. [ 11 ] utilized PointNet to abstract multi-scale features from raw point clouds by stacking set abstraction layers. This can be considered as an increasing path of “receptive fields”, with the common disadvantage of a loss of finer local geometry in each hierarchical layer.…”

Section: Related Workmentioning

confidence: 99%

“…In this paper, we follow [ 11 ] by consuming pure 3D point cloud coordination as the only data while alternatively assuming expression samples to be geometric objects lying on characterized low-dimensional manifolds. To accomplish this idea, we apply the geometric wavelet scattering transform [ 12 ] in 3D FER in a novel way; our experimental results on the Bosphorus [ 13 ] and 3D-BUFE [ 14 ] datasets prove the high efficiency of this approach in terms of balancing expressiveness with stability against multiple random deformation or noise.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Facial Expression Recognition with Geometric Scattering on 3D Point Clouds

Cheng

et al. 2022

Sensors

View full text Add to dashboard Cite

As one of the pioneering data representations, the point cloud has shown its straightforward capacity to depict fine geometry in many applications, including computer graphics, molecular structurology, modern sensing signal processing, and more. However, unlike computer graphs obtained with auxiliary regularization techniques or from syntheses, raw sensor/scanner (metric) data often contain natural random noise caused by multiple extrinsic factors, especially in the case of high-speed imaging scenarios. On the other hand, grid-like imaging techniques (e.g., RGB images or video frames) tend to entangle interesting aspects with environmental variations such as pose/illuminations with Euclidean sampling/processing pipelines. As one such typical problem, 3D Facial Expression Recognition (3D FER) has been developed into a new stage, with remaining difficulties involving the implementation of efficient feature abstraction methods for high dimensional observations and of stabilizing methods to obtain adequate robustness in cases of random exterior variations. In this paper, a localized and smoothed overlapping kernel is proposed to extract discriminative inherent geometric features. By association between the induced deformation stability and certain types of exterior perturbations through manifold scattering transform, we provide a novel framework that directly consumes point cloud coordinates for FER while requiring no predefined meshes or other features/signals. As a result, our compact framework achieves 78.33% accuracy on the Bosphorus dataset for expression recognition challenge and 77.55% on 3D-BUFE.

show abstract

Section: Resultsmentioning

confidence: 99%