Feature level fusion framework for multimodal biometric system based on CCA with SVM classifier and cosine similarity measure

Kamlaskar, Chetana; Abhyankar, Aditya

doi:10.1080/1448837x.2022.2129147

Cited by 2 publications

(1 citation statement)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Feature-level fusion works with the idea of extracting the most discriminating features from the extracted features and removing redundant information [31,32]. It works with the conjecture of strict time synchronicity between different modalities and performs distinctively in cases where the modalities differ considerably in temporal characteristics [31,33].…”

Section: Feature-level Multimodal Fusionmentioning

confidence: 99%

Review on Multimodal Fusion Techniques for Human Emotion Recognition

Karani¹,

Desai²

2022

IJACSA

View full text Add to dashboard Cite

Emotions play an essential role in human life for planning and decision making. Emotion identification and recognition is a widely explored field in the area of artificial intelligence and affective computing as a means of empathizing with humans and thereby improving human machine interaction. Though audio visual cues are vital for recognizing human emotions, they are sometimes insufficient in identifying emotions of people who are good at hiding emotions or people suffering from Alexithymia. Considering other dimensions like Electroencephalogram (EEG) or text, along with audio visual cues can aid in improving the results in such situations. Taking advantage of the complementarity of multiple modalities normally helps capture emotions more accurately compared to single modality. However, to achieve precise and accurate results, correct fusion of these multimodal signals is solicited. This study provides a detailed review of different multimodal fusion techniques that can be used for emotion recognition. This paper proposes in-depth study of feature-level fusion, decision-level fusion and hybrid fusion techniques for identifying human emotions based on multimodal inputs and compare the results. The study concentrates on three different modalities i.e., facial images, audio and text for experimentation; at least one of which differs in temporal characteristics. The result suggests that hybrid fusion works best in combining multiple modalities which differ in time synchronicity.

show abstract

Section: Feature-level Multimodal Fusionmentioning

confidence: 99%

Review on Multimodal Fusion Techniques for Human Emotion Recognition

Karani¹,

Desai²

2022

IJACSA

View full text Add to dashboard Cite

show abstract

Enhancing Biometric Security with Bimodal Deep Learning and Feature-level Fusion of Facial and Voice Data

Merit,

Beladgham

2024

JTIT

View full text Add to dashboard Cite

Recent research in biometric technologies underscores the benefits of multimodal systems that use multiple traits to enhance security by complicating the replication of samples from genuine users. To address this, we present a bimodal deep learning network (BDLN or BNet) that integrates facial and voice modalities. Voice features are extracted using the SincNet architecture, and facial image features are obtained from convolutional layers. Proposed network fuses these feature vectors using either averaging or concatenation methods. A dense connected layer then processes the combined vector to produce a dual-modal vector that encapsulates distinctive user features. This dual-modal vector, processed through a softmax activation function and another dense connected layer, is used for identification. The presented system achieved an identification accuracy of 99% and a low equal error rate (EER) of 0.13% for verification. These results, derived from the VidTimit and BIOMEX-DB datasets, highlight the effectiveness of the proposed bimodal approach in improving biometric security.

show abstract

Feature level fusion framework for multimodal biometric system based on CCA with SVM classifier and cosine similarity measure

Cited by 2 publications

References 46 publications

Review on Multimodal Fusion Techniques for Human Emotion Recognition

Review on Multimodal Fusion Techniques for Human Emotion Recognition

Enhancing Biometric Security with Bimodal Deep Learning and Feature-level Fusion of Facial and Voice Data

Contact Info

Product

Resources

About