No abstract

Nowadays, hand gesture recognition has become an alternative for human-machine interaction. It has covered a large area of applications like 3D game technology, sign language interpreting, VR (virtual reality) environment, and robotics. But detection of the hand portion has become a challenging task in computer vision and pattern recognition communities. Deep learning algorithm like convolutional neural network (CNN) architecture has become a very popular choice for classification tasks, but CNN architectures suffer from some problems like high variance during prediction, overfitting problem and also prediction errors. To overcome these problems, an ensemble of CNN-based approaches is presented in this paper. Firstly, the gesture portion is detected by using the background separation method based on binary thresholding. After that, the contour portion is extracted, and the hand region is segmented. Then, the images have been resized and fed into three individual CNN models to train them in parallel. In the last part, the output scores of CNN models are averaged to construct an optimal ensemble model for the final prediction. Two publicly available datasets (labeled as Dataset-1 and Dataset-2) containing infrared images and one self-constructed dataset have been used to validate the proposed system. Exper-

No abstract

In this work, a real-time hand gesture recognition system-based human-computer interface (HCI) is presented. The system consists of six stages: (1) hand detection, (2) gesture segmentation, (3) use of six pre-trained CNN models by using the transfer-learning method, (4) building an interactive human-machine interface, (5) development of a gesture-controlled virtual mouse, (6) use of Kalman filter to estimate the hand position, based on that the smoothness of the motion of pointer is improved. Six pre-trained convolutional neural network (CNN) models such as VGG16, VGG19, ResNet50, ResNet101, Inception-V1, and MobileNet-V1 have been used to classify hand gesture images. Three multi-class datasets (two publicly and one custom) have been used to validate the models. Considering the models' performances, it is observed that Inception-V1 has significantly shown a better classification performance compared to the other five pre-trained models in terms of accuracy, precision, recall, and F-score values. The gesture recognition system is expanded and used to control some desktop applications (such as VLC player, audio player, file management, playing 2D Super-Mario-Bros game, etc.) with different customized gesture commands in real-time scenarios. The average speed of this system has reached 35 fps (frame per second), which meets the requirements for the real-time scenario. We have also shown the performance analysis of gesture-controlled applications and obtained the average response time (ms) for each control of every application in the real-time scenario. This model will be beneficial for physically disabled people interacting with the desktops.