The version in the Kent Academic Repository may differ from the final published version. Users are advised to check http://kar.kent.ac.uk for the status of the paper. Users should always cite the published version of record.

Human activity and gesture recognition is an important component of rapidly growing domain of ambient intelligence, in particular in assisting living and smart homes. In this paper, we propose to combine the power of two deep learning techniques, the convolutional neural networks (CNN) and the recurrent neural networks (RNN), for automated hand gesture recognition using both depth and skeleton data. Each of these types of data can be used separately to train neural networks to recognize hand gestures. While RNN were reported previously to perform well in recognition of sequences of movement for each skeleton joint given the skeleton information only, this study aims at utilizing depth data and apply CNN to extract important spatial information from the depth images. Together, the tandem CNN+RNN is capable of recognizing a sequence of gestures more accurately. As well, various types of fusion are studied to combine both the skeleton and depth information in order to extract temporal-spatial information. An overall accuracy of 85.46% is achieved on the dynamic hand gesture-14/28 dataset.

Facial expressions are a valuable source of information that accompanies facial biometrics. Early detection of physiological and psycho-emotional data from facial expressions is linked to the situational awareness module of any advanced biometric system for personal state re/identification. In this article, a new method that utilizes both texture and geometric information of facial fiducial points is presented. We investigate Gauss-Laguerre wavelets, which have rich frequency extraction capabilities, to extract texture information of various facial expressions. Rotation invariance and the multiscale approach of these wavelets make the feature extraction robust. Moreover, geometric positions of fiducial points provide valuable information for upper/lower face action units. The combination of these two types of features is used for facial expression classification. The performance of this system has been validated on three public databases: the JAFFE, the Cohn-Kanade, and the MMI image.

This paper describes a prototype of an intelligent Stress Monitoring Assistant (SMA), -the next generation of stress detectors. The SMA is intended for the first responders and professionals coping with exposure to extreme physical and psychological stressors, e.g. firefighters, combat military personnel, explosive ordnance disposal operatives, law enforcement officers, emergency medical technicians, and paramedics. Stress impacts human behavior and decision-making, which can be propagated between the team members. The SMA is an integral part of the Decision Support System, it is a component of the decision support perception-action cycle. We model this cycle as a cognitive dynamic system. The intelligent part of the SMA is designed using a) a residual-temporal convolution network for learning data from sensors and detection of stress features, and b) a reasoning mechanism based on a causal network for fusion at various levels. The SMA prototype has been tested using a multi-factor physiological dataset WEarable Stress and Affect Detection (WESAD). In both modes, the stress recognition and stress detection, the SMA achieves an accuracy of 86% and 98% for the WESAD dataset, respectively. This performance is superior to the known results in satisfying the requirements of reliable decision support.

This paper revisits the concept of an authentication machine (A-machine) that aims at identifying/verifying humans. Although A-machines in the closed-set application scenario are well understood and commonly used for access control utilizing human biometrics (face, iris, and fingerprints), open-set applications of Amachines have yet to be equally characterized. This paper presents an analysis and taxonomy of A-machines, trends, and challenges of open-set real-world applications. This paper makes the following contributions to the area of open-set A-machines: 1) a survey of applications; 2) new novel life cycle metrics for theoretical, predicted, and operational performance evaluation; 3) a new concept of evidence accumulation for risk assessment; 4) new criteria for the comparison of A-machines based on the notion of a supporting assistant; and 5) a new approach to border personnel training based on the A-machine training mode. It offers a technique for modeling A-machines using belief (Bayesian) networks and provides an example of this technique for biometric-based e-profiling.