Implementation of real-time static hand gesture recognition using artificial neural network

Abstract

This paper implements static hand gesture recognition in recognizing the alphabetical sign from "A" to "Z", number from "0" to "9", and additional punctuation mark such as "Period", "Question Mark", and "Space" in Sistem Isyarat Bahasa Indonesia (SIBI). Hand gestures are obtained by evaluating the contour representation from image segmentation of the glove wore by user. Then, it is classified using Artificial Neural Network (ANN) based on the training model previously built from 100 images for each gesture. The accuracy rate of hand gesture translation is calculated to be 90%. Moreover, speech translation recognizes NATO phonetic letter as the speech input for translation.

A systematic review on hand gesture recognition techniques, challenges and applications

“…Results showed that the vision-based technique was more stable and reliable compared to the data glove-based technique. Hand gestures are obtained by evaluating the contour captured from the image segmentation using a glove worn by the speaker in Rosalina et al (2017). Also, in Danling, Yuanlong & Huaping (2016) they used a novel data glove called YoBu to collect data for gesture recognition.…”

“…The work of Haitham, Alaa & Sabah (2017), Weiguo et al (2017), Rosalina et al (2017), , , Shunzhan et al (2017), Erhan, Hakan & Baran (2017), Vladislava, Predrag & Goran (2016), Deepali & Milind (2016) and Ananta & Piyanuch (2016) proposed using Artificial Neural Networks (ANN) for classification, whereas in and SoftMax output layer was used with feedforward ANN; in Shunzhan et al (2017), Erhan, Hakan & Baran (2017), Alvi, Fatema & Mohammad (2016), Vladislava, Predrag & Goran (2016); Deepali & Milind (2016) and Ananta & Piyanuch (2016) backpropagation training methods were used, and the authors in Jessie et al (2016) used Kohonen selforganizing maps as a type of ANN to classify data sets in unsupervised manner to convert hand gestures into Filipino words.…”

“…An alternative to talking for deaf & dumb people who have hearing or speech problems using sign language recognition and hand gestures was proposed in Himadri et al (2018), Oinam et al (2017), Reshna & Jayaraju (2017), Vivek, Ramesh & Kagalkar (2017), Oyndrila et al (2016), Ashish & Aarti (2016a), Ashish & Aarti (2016b) and Rishabh et al (2016). Whereas in Rosalina et al (2017) a sign language that consists of the alphabetic from ''A'' to ''Z'', the numbers from ''0'' to ''9'', and some additional punctuation marks like ''Period'', ''Question Mark'', and ''Space'' using static hand gesture recognition was produced.…”

“…The movements of a hand on the air as shown in Fig. 2 (Rosalina et al, 2017), is an example of a sign language using hand gestures. The representation of hand gestures is varying from reflecting a certain action to delivering a message that actually has a meaning (Adam, 2004).…”

Indonesia Sign Language Recognition using Convolutional Neural Network

“…Meanwhile, Khotimah et al implemented weighted k-nearest neighbor classification for dynamic sign language recognition [11]. Rosalina et al used artificial intelligence to recognize SIBI [12]. Other studies utilized Hidden Markov Model [13] and Naïve Bayes [14] methods.…”