Human Micro-Expressions in Multimodal Social Behavioral Biometrics

Abstract

The advent of Social Behavioral Biometrics (SBB) in the realm of person identification has underscored the importance of understanding unique patterns of social interactions and communication. This paper introduces a novel multimodal SBB system that integrates human micro-expressions from text, an emerging biometric trait, with other established SBB traits in order to enhance online user identification performance. Including human micro-expression, the proposed method extracts five other original SBB traits for a comprehensive representation of the social behavioral characteristics of an individual. Upon finding the independent person identification score by every SBB trait, a rank-level fusion that leverages the weighted Borda count is employed to fuse the scores from all the traits, obtaining the final identification score. The proposed method is evaluated on a benchmark dataset of 250 Twitter users, and the results indicate that the incorporation of human micro-expression with existing SBB traits can substantially boost the overall online user identification performance, with an accuracy of 73.87% and a recall score of 74%. Furthermore, the proposed method outperforms the state-of-the-art SBB systems.