Sign Language Recognition Using Motion History Volume and Hybrid Neural Networks

Abstract

Abstract-In this paper, we present a sign language recognition model which does not use any wearable devices for object tracking. The system design issues and implementation issues such as data representation, feature extraction and pattern classification methods are discussed. The proposed data representation method for sign language patterns is robust for spatio-temporal variances of feature points. We present a feature extraction technique which can improve the computation speed by reducing the amount of feature data. A neural network model which is capable of incremental learning is introduced. We have defined a measure which reflects the relevance between the feature types and the pattern classes. The measure makes it possible to select more effective features without any degradation of performance. Through the experiments using six types of sign language patterns, the proposed model is evaluated empirically.

Indonesian Sign Language (BISINDO) Recognition Using Accurate and Fast Dynamic Time Warping Learning Model

“…Based on the results obtained in this paper and previous studies related to BISINDO recognition, we decided to use AF-DTW model for our next research in recognizing sentence of BISINDO. However, some deep learning algorithms can be tried to improve the accuracy of BISINDO recognition, as in [19], [20]. In fact, the sign language is used by the deaf or hard-hearing people to communicate in the form of sentence, not a word.…”

A Real-time Hand Gesture Recognition Technique and Its Application to Music Display System

“…Therefore, each sub-layer generates a feature map which reflects successively larger ranges of the preceding unit. In our previous study [7], we introduced an extended version of the CNN for temporal feature extraction. The input data for the feature extractor are represented as a spatiotemporal volume which is described in the previous section.…”