Multimodal Biometrics Recognition from Facial Video via Deep Learning

Maity, Sayan; Abdel-Mottaleb, Mohamed; Asfour, Shihab

doi:10.5121/sipij.2017.8101

Cited by 12 publications

(16 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides, the form and motion are natural when obtaining the dynamic face stimuli from the videos. Findings from these past studies imply that the advantage brought by the dynamic face stimuli is more obvious in subtle expressions than elementary expressions [4,11,18]. In 2017, findings in [18] show that using video database (e.g., Honda/UCSD) for face recognition is able to achieve a recognition rate as high as 97.14% via deep learning.…”

Section: Existing Video-based Face/expression Recognitionmentioning

confidence: 99%

Faceless identification based on temporal strips

Leong

Phan

Baskaran

et al. 2020

Multimed Tools Appl

View full text Add to dashboard Cite

This paper first presents a novel approach for modelling facial features, Local Directional Texture (LDT), which exploits the unique directional information in image textures for the problem of face recognition. A variant of LDT with privacy-preserving temporal strips (TS) is then considered to achieve faceless recognition with a higher degree of privacy while maintaining high accuracy. The TS uses two strips of pixel blocks from the temporal planes, XT and YT, for face recognition. By removing the reliance on spatial context (i.e., XY plane) for this task, the proposed method withholds facial appearance information from public view, where only one-dimensional temporal information that varies across time are extracted for recognition. Thus, privacy is assured, yet without impeding the facial recognition task which is vital for many security applications such as street surveillance and perimeter access control. To validate the reliability of the proposed method, experiments were carried out using the Honda/UCSD, CK+, CAS(ME) 2 and CASME II databases. The proposed method achieved a recognition rate of 98.26% in the standard video-based face recognition database, Honda/UCSD. It also offers a 81.92% reduction in the dimension length required for storing the extracted features, in contrast to the conventional LBP-TOP.

show abstract

Section: Existing Video-based Face/expression Recognitionmentioning

confidence: 99%

Faceless identification based on temporal strips

Leong

Phan

Baskaran

et al. 2020

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…In [3], the outputs of non-homogeneous classifiers, which are developed based on acoustic features from voice and visual features from face, are fused at the hybrid rank/measurement level to improve the identification rate of the system. Deep learning algorithms have also been used to address the problem of face recognition and action recognition, respectively [14,15]. Despite the fact that the above-mentioned studies are non-invasive multimodal biometric identification systems, the fusion methods that are employed in these systems require the concurrent presence of all biometric modalities for proper functioning, whereas the architecture that is reported in this paper relaxes this condition.…”

Section: Related Studiesmentioning

confidence: 99%

“…Here, we use 32-compnenent GMM to build the UBM. The UBM is represented by a GMM with 32-compnents, as denoted by λ UBM , that characterized by its probability density function as (14).…”

Section: Dedicated Processing Units For Voice-based Feature Vectormentioning

confidence: 99%

A Multi-Modal Person Recognition System for Social Robots

Al-Qaderi

Rad

2018

Applied Sciences

View full text Add to dashboard Cite

The paper presents a solution to the problem of person recognition by social robots via a novel brain-inspired multi-modal perceptual system. The system employs spiking neural network to integrate face, body features, and voice data to recognize a person in various social human-robot interaction scenarios. We suggest that, by and large, most reported multi-biometric person recognition algorithms require active participation by the subject and as such are not appropriate for social human-robot interactions. However, the proposed algorithm relaxes this constraint. As there are no public datasets for multimodal systems, we designed a hybrid dataset by integration of the ubiquitous FERET, RGB-D, and TIDIGITS datasets for face recognition, person recognition, and speaker recognition, respectively. The combined dataset facilitates association of facial features, body shape, and speech signature for multimodal person recognition in social settings. This multimodal dataset is employed for testing the algorithm. We assess the performance of the algorithm and discuss its merits against related methods. Within the context of the social robotics, the results suggest the superiority of the proposed method over other reported person recognition algorithms.

show abstract

“…Deep convolutional neural networks have produced state-of-the-art results on various benchmarks [1], [2]. Many Researches in the field of convolutional neural networks, practically proved that deeper networks have higher accuracy.…”

Section: Introductionmentioning

confidence: 99%

Test-cost-sensitive Convolutional Neural Networks with Expert Branches

Naghibi¹,

Anvari²,

Forghani³

et al. 2019

SIPIJ

View full text Add to dashboard Cite

It has been proven that deeper convolutional neural networks (CNN) can result in better accuracy in many problems, but this accuracy comes with a high computational cost. Also, input instances have not the same difficulty. As a solution for accuracy vs. computational cost dilemma, we introduce a new test-cost-sensitive method for convolutional neural networks. This method trains a CNN with a set of auxiliary outputs and expert branches in some middle layers of the network. The expert branches decide to use a shallower part of the network or going deeper to the end, based on the difficulty of input instance. The expert branches learn to determine: is the current network prediction is wrong and if the given instance passed to deeper layers of the network it will generate right output; If not, then the expert branches stop the computation process. The experimental results on standard dataset CIFAR-10 show that the proposed method can train models with lower test-cost and competitive accuracy in comparison with basic models.

show abstract

Multimodal Biometrics Recognition from Facial Video via Deep Learning

Abstract: Biometrics identification using multiple modalities has attracted the attention of many researchers as it

Cited by 12 publications

References 6 publications

Faceless identification based on temporal strips

Faceless identification based on temporal strips

A Multi-Modal Person Recognition System for Social Robots

Test-cost-sensitive Convolutional Neural Networks with Expert Branches

Contact Info

Product

Resources

About