PointNet++ and Three Layers of Features Fusion for Occlusion Three-Dimensional Ear Recognition Based on One Sample per Person

Abstract

The ear’s relatively stable structure makes it suitable for recognition. In common identification applications, only one sample per person (OSPP) is registered in a gallery; consequently, effectively training deep-learning-based ear recognition approach is difficult. The state-of-the-art (SOA) 3D ear recognition using the OSPP approach bottlenecks when large occluding objects are close to the ear. Hence, we propose a system that combines PointNet++ and three layers of features that are capable of extracting rich identification information from a 3D ear. Our goal is to correctly recognize a 3D ear affected by a large nearby occlusion using one sample per person (OSPP) registered in a gallery. The system comprises four primary components: (1) segmentation; (2) local and local joint structural (LJS) feature extraction; (3) holistic feature extraction; and (4) fusion. We use PointNet++ for ear segmentation. For local and LJS feature extraction, we propose an LJS feature descriptor–pairwise surface patch cropped using a symmetrical hemisphere cut-structured histogram with an indexed shape (PSPHIS) descriptor. Furthermore, we propose a local and LJS matching engine based on the proposed LJS feature descriptor and SOA surface patch histogram indexed shape (SPHIS) local feature descriptor. For holistic feature extraction, we use a voxelization method for global matching. For the fusion component, we use a weighted fusion method to recognize the 3D ear. The experimental results demonstrate that the proposed system outperforms the SOA normalization-free 3D ear recognition methods using OSPP when the ear surface is influenced by a large nearby occlusion.

Corresponding keypoint constrained sparse representation three‐dimensional ear recognition via one sample per person

“…Third, the ICP method [23] must search the entire surface of the gallery ear image for the point closest to the point from the probe sample. Furthermore, feature fusion method [31][32][33]38], surface variation feature extraction method [21], and index method [40] must search the entire surface of the gallery ear image for the match of the query feature. Consequently, these methods require more time to search the matching feature than the proposed system.…”

“…However, when these indicators are used to evaluate the LC_KSVD method [24], which is difficult to run on the OSPP gallery, it is difficult to use the running time of the LC_KSVD method applied to the gallery composed of multiple samples of each subject to represent the running time of the LC_KSVD method applied to the OSPP gallery. Moreover, when applied to methods [7, 21, 23, 31–33, 38, 40], the time required of one identification operation is easily affected by the scale of the OSPP gallery, that is, with the expansion of the scale of the OSPP gallery, the time required of one identification operation is easy to increase. To solve these problems, we propose an indicator: the time of one identification operation shared by each gallery subject (TOIOS‐EGS).…”

“…Hence, for this method [24], TOIOS‐EGS obtained using a gallery containing 762 samples of 127 subjects was considered to be equal to that obtained using an OSPP gallery. Furthermore, the number of features considered in previous works [7, 21, 23, 31–33, 38, 40] and in our CKC‐SR methods required to match is approximately proportional to the number of subjects that comprise the OSPP gallery. Hence, when applied to the OSPP gallery with different scales, the TOIOS‐EGS of any among these methods does not show any remarkable difference.…”

Learning Deformable Network for 3D Object Detection on Point Clouds

“…Similarly, SECOND [28] applies sparse convolution to get efficient features from voxels. F-PointNet [29] uses a 2D object detector on the image to make 2D object proposal first and then get corresponding frustum point cloud as the base of regression and prediction, while the accuracy of F-PointNet relies too much on the 2D detector.…”

“…( 27) else (28) add new voxel in hash table. (29) end if (30) end if (31) end for ALGORITHM 1: Voxelization algorithm. Mobile Information Systems form a square receptive field, the introduction of deformable convolution can better adapt to the changes of different directions of vehicles.…”