Hybrid Score- and Rank-Level Fusion for Person Identification using Face and ECG Data

Abstract

Uni-modal identification systems are vulnerable to errors in sensor data collection and are therefore more likely to misidentify subjects. For instance, relying on data solely from an RGB face camera can cause problems in poorly lit environments or if subjects do not face the camera. Other identification methods such as electrocardiograms (ECG) have issues with improper lead connections to the skin. Errors in identification are minimized through the fusion of information gathered from both of these models. This paper proposes a methodology for combining the identification results of face and ECG data using Part A of the BioVid Heat Pain Database containing synchronized RGB-video and ECG data on 87 subjects. Using 10-fold cross-validation, face identification was 98.8% accurate, while the ECG identification was 96.1% accurate. By using a fusion approach the identification accuracy improved to 99.8%. Our proposed methodology allows for identification accuracies to be significantly improved by using disparate face and ECG models that have non-overlapping modalities.

A Comprehensive Survey on ECG Signals as New Biometric Modality for Human Authentication: Recent Advances and Future Challenges

“…The fusion strategy of the information of multimodal Biometric modalities is achieved in several methods, which are categorized based on parameters like fusion scheme, information sources, and fusion levels [394].…”

“…(a) Fusion Scheme: Sequential and parallel fusion are the two different types of topological multimodal biometric fusion techniques [394]. While multimodal biometric modalities are processed simultaneously in parallel fusion techniques, they are processed in a sequential top-down merge technique until an acceptable match is obtained.…”

“…While multimodal biometric modalities are processed simultaneously in parallel fusion techniques, they are processed in a sequential top-down merge technique until an acceptable match is obtained. (b) Information Sources [394]: The information for multimodal biometric modalities can be achieved by [395], [396], such as data-level, feature-level, score-level, rank-level, and decision-level, the fusion at the match score level has been widely investigated in the literature. We briefly highlight some of the commonly used fusion levels with more emphasis on the score level:…”

“…• Feature-level [397]: Here, the feature sets arising from multiple biometric sources are fused into a single feature set via proper feature normalization, transformation, and reduction techniques. However, feature-level fusion, the feature sets arising from multiple biometric sources may not be readily fused [394], i.e., one-dimensional signals from an ECG and two-dimensional images from a camera. • Score-level [396]: Match score-level fusion comprises fusing the match scores generated by multiple classifiers (or matchers) to furnish a decision regarding the identity of the subject.…”

Variation-Stable Fusion for PPG-Based Biometric System

“…Different score fusion methods were simulated where the weighted sum rule performed the best. Another work by [13] studied a multimodal system consisting of the face and ECG signal for user identification. A subsequent work in [14] learned features from the fingerprints and ECG using convolutional neural network (CNN) in parallel.…”