Efficient Kinect Sensor-based Kurdish Sign Language Recognition Using Echo System Network

Abstract

Sign language assists in building communication and bridging gaps in understanding. Automatic sign language recognition (ASLR) is a field that has recently been studied for various sign languages. However, Kurdish sign language (KuSL) is relatively new and therefore researches and designed datasets on it are limited. This paper has proposed a model to translate KuSL into text and has designed a dataset using Kinect V2 sensor. The computation complexity of feature extraction and classification steps, which are serious problems for ASLR, has been investigated in this paper. The paper proposed a feature engineering approach on the skeleton position alone to provide a better representation of the features and avoid the use of all of the image information. In addition, the paper proposed model makes use of recurrent neural networks (RNNs)-based models. Training RNNs is inherently difficult, and consequently, motivates to investigate alternatives. Besides the trainable long short-term memory (LSTM), this study has proposed the untrained low complexity echo system network (ESN) classifier. The accuracy of both LSTM and ESN indicates they can outperform those in state-of-the-art studies. In addition, ESN which has not been proposed thus far for ASLT exhibits comparable accuracy to the LSTM with a significantly lower training time.

Effective Kurdish Sign Language Detection and Classification Using Convolutional Neural Networks

“…Moreover, the ResNet (CNN) model [25] is located in the lower position in terms of classes and has 5 classes with a 78.49% accuracy rate, followed by the ANN model [32] with 10 classes and 98% accuracy. In contrast, the RNN model [33] classi ed 84 classes of Kurdish body sign language with an accuracy of 97.4%. The model was trained on a dataset that was captured by Kinect.…”

“…The system's structure, whether it deals with static language signs expressed in a single picture or dynamic language signs portrayed in a series of images, on one side and/or continuous input on the other, manages the data gathering technique, feature extraction, and data classi cation. Because static signs are not time series, they can be simply expressed in a global feature [33]. According to [26] The Persian sign language (PSL) system applies wavelet transform and neural networks (NN) to recognize static gestures of the Persian alphabet.…”

“…Recently, Several studies have been performed on several sign languages, including ASL [13][14] [15][16] [17] [18] [19][20], BSL [21]. Arabic sign language (ArSL) [1][22] [23], Turkish sign language [24][25],Persian sign language [26] [27] [28], Indian sign language [29] [30], and to the best of the authors' knowledge, the only three available researches have been on KuSL, including 12 classes [31], 10 classes [32], and 84 classes [33].…”

Improved Recognition of Kurdish Sign Language Using Modified CNN

“…Numerous studies on variant SL, such as ASL [17][18][19][20][21][22][23][24][25], BSL [26], Arabic SL [1,27,28], Turkish SL [29,30], Persian SL [31][32][33][34], Indian SL [35,36], and others, have been carried out in recent years. To the best of our knowledge, the only accessible studies focused on KuSL and consisted of 12 classes [37], 10 classes [38], and 84 classes [39].…”

Human Body Posture Recognition Approaches

“…In the final step, they are categorized using the model estimation of the main points of the body through the neural network (Gupta, et al, 2020). Collecting data in the vision-based method utilize one or more conventional cameras and specialized cameras for depth images (Mirza and Al-Talabani, 2021). The visionbased system has the benefit of not requiring users to wear any sensors, but its performance is heavily influenced by visual angle, lighting conditions, and other environmental variables.…”