Trajectory-based recognition of dynamic Persian sign language using hidden Markov model

Abstract

Sign Language Recognition (SLR) is an important step in facilitating the communication among deaf people and the rest of society. Existing Persian sign language recognition systems are mainly restricted to static signs which are not very useful in everyday communications.In this study, a dynamic Persian sign language recognition system is presented. A collection of 1200 videos were captured from 12 individuals performing 20 dynamic signs with a simple white glove. The trajectory of the hands, along with hand shape information were extracted from each video using a simple region-growing technique. These time-varying trajectories were then modeled using Hidden Markov Model (HMM) with Gaussian probability density functions as observations. The performance of the system was evaluated in different experimental strategies. Signer-independent and signer-dependent experiments were performed on the proposed system and the average accuracy of 97.48% was obtained. The experimental results demonstrated that the performance of the system is independent of the subject and it can also perform excellently even with a limited number of training data. are manual gestures, while non-manual gestures like head movements (e.g. nodding), body movement (e.g. shrugging) and face expressions also play an important role in sign communications. Unfortunately, the use of sign language is usually restricted to the deaf community resulting in restricted communication of them with the rest of the world. Therefore, they are usually excluded from society and deprived of their rights to have equal educational and career opportunities. In order to address this problem, Sign Language Recognition (SLR) systems are developed to translate this language into speech or text. Developing efficient SLR systems can facilitate the communication of deaf people in society and remove the barriers for them.One of the basic issues regarding SLR is that there is not a universal sign language.Sign languages of different countries have their own grammar rules. SLR systems have been rapidly developed in recent years for different sign languages including American [1-3], Chinese [4], Australian [5], Arabic [6,7], Indian [8], Spanish [9] and Japanese [10]. For more reviews on sign language and different approaches developed for SLR systems refer to [11,12].Due to its broad range of capabilities, Machine Vision (MV) is the major tool used in the development of SLR systems. An MV-based SLR system usually consists of three components: hand tracker, feature extractor and classifier. Hand trackers job is to segment hand regions from the background of the input video frames. Some studies rely on data gloves to track the hand movements for hand tracking [6,[13][14][15][16]. Although these gloves are

Static and Dynamic Hand Gesture Recognition Using GIST and Linear Discriminant Analysis

“…The literature [23,24] implemented end-to-end hand gesture recognition system by using convolutional neural network (CNN) with any auxiliary method for pre-processing purpose. Hoang [27]. Kraljevic` et al, (2020) implemented Croatian sign language recognition system by using end-to-end process of deep learning.…”

A State-of-the-Art Survey on Face Recognition Methods

“…HMM is a stochastic model which contains a Markov chain with an imperceptible or hidden grouping of states (Azar & Seyedarabi, 2019). In this approach, the observation vectors sequence of face images is described using their statistical distribution.…”

“…Azar S. G. et al (Azar & Seyedarabi, 2019) introduced a Dynamic Persian sign language recognition system. Hand trajectories alongside three-hand shape information were separated from video frames utilizing a region thriving method.…”

Analysis and Prediction of Pedestrians’ Violation Behavior at the Intersection Based on a Markov Chain

“…Mihiran et al [6] proposed a combination of a Markov model and a Bayesian network for a real-time monitoring method to estimate the probability of pedestrian group violations. Saeideh et al [7] applied a region growing technique to construct a Hidden Markov Model (HMM) to simulate the time-varying trajectory of moving objects. Javan D. et al [8] analyzed polydisperse systems by a Markov chain algorithm, but ignored the influence of interaction between groups.…”