Real-Time Turkish Sign Language Recognition Using Cascade Voting Approach with Handcrafted Features

Abstract

In this study, a machine learning-based system, which recognises the Turkish sign language person-independent in real-time, was developed. A leap motion sensor was used to obtain raw data from individuals. Then, handcraft features were extracted by using Euclidean distance on the raw data. Handcraft features include finger-to-finger, finger -to-palm, finger -to-wrist bone, palm-to-palm and wrist-to-wrist distances. LR, k-NN, RF, DNN, ANN single classifiers were trained using the handcraft features. Cascade voting approach was applied with two-step voting. The first voting was applied for each classifier’s final prediction. Then, the second voting, which voted the prediction of all classifiers at the final decision stage, was applied to improve the performance of the proposed system. The proposed system was tested in real-time by an individual whose hand data were not involved in the training dataset. According to the results, the proposed system presents 100 % value of accuracy in the classification of one hand letters. Besides, the recognition accuracy ratio of the system is 100 % on the two hands letters, except “J” and “H” letters. The recognition accuracy rates were 80 % and 90 %, respectively for “J” and “H” letters. Overall, the cascade voting approach presented a high average classification performance with 98.97 % value of accuracy. The proposed system enables Turkish sign language recognition with high accuracy rates in real time.

Improved Recognition of Kurdish Sign Language Using Modified CNN

“…Numerous studies on variant SL, such as ASL [17][18][19][20][21][22][23][24][25], BSL [26], Arabic SL [1,27,28], Turkish SL [29,30], Persian SL [31][32][33][34], Indian SL [35,36], and others, have been carried out in recent years. To the best of our knowledge, the only accessible studies focused on KuSL and consisted of 12 classes [37], 10 classes [38], and 84 classes [39].…”

Effective Kurdish Sign Language Detection and Classification Using Convolutional Neural Networks

“…Recently, Several studies have been performed on several sign languages, including ASL [13][14] [15][16] [17] [18] [19][20], BSL [21]. Arabic sign language (ArSL) [1][22] [23], Turkish sign language [24][25],Persian sign language [26] [27] [28], Indian sign language [29] [30], and to the best of the authors' knowledge, the only three available researches have been on KuSL, including 12 classes [31], 10 classes [32], and 84 classes [33].…”

A Real-Time Dynamic Gesture Variability Recognition Method Based on Convolutional Neural Networks

“…Modern research results make it clear that machine learning methods [8][9][10][11][12][13][14][15][16] based on deep neural networks [8][9][10][11][12][13][14][15], in comparison with traditional classical approaches [7,16,17], which are based on linear classifiers (e.g., the support vector method (SVM) or hidden Markov classification (HMC)), can demonstrate good results in segmentation, classification, and recognition as static and dynamic sign language (SL) elements. Thus, with the help of two-threaded convolutional neural networks [2,9,10], it is possible to extract the spatiotemporal features of a gesture from full-color images (RGB format) and 3D frames (depth map) of video streams separately.…”

“…Thus, with the help of two-threaded convolutional neural networks [2,9,10], it is possible to extract the spatiotemporal features of a gesture from full-color images (RGB format) and 3D frames (depth map) of video streams separately. In turn, a deep neural network (DNN) is used to perform segmentation and classification of the hand shape with multiple architectures and sizes of the input images [4,11]. In addition, it was revealed that the architecture of the long short-term memory (LSTM) neural networks [1,3,12,13,15,[18][19][20], with the help of long and short-term memory, can extract the spatiotemporal characteristics of a gesture from sequences of previously annotated 2D regions with a gesture.…”