In-Air Hand Gesture Signature Recognition Using Multi-Scale Convolutional Neural Networks

Abstract

The hand signature is a unique handwritten name or symbol that serves as a proof of identity. Due to its practicality and widespread use, hand signature is still used by financial institutions as a means of verifying and validating the identity of their customers. The emergence of the COVID-19 global pandemic has raised hygiene concerns regarding the conventional touch-based hand signature recognition system, which often requires sharing the acquisition devices among the public. This paper presents in-air hand gesture signature recognition using convolutional neural networks to address this concern. We designed a shallow multi-scale convolutional neural network using 3x3 and 5x5 kernel filter sizes to extract features on different scales. The feature maps from these two filters are then concatenated to provide more robust features, which improve the model’s performance. The experiment results show that the proposed architecture outperforms other architectures, which obtained the highest accuracy of 93.00%. On the other hand, our architecture consumed significantly fewer computational resources, requiring only an average of 3 minutes and 33 seconds to train. Additionally, the performance of the proposed architecture could be further enhanced by integrating it with recurrent neural networks (RNN). This integrated architecture of convolutional recurrent neural networks (C-RNN) can capture spatio-temporal features simultaneously.