Sign language localization: Learning to eliminate language dialects

Abstract

Machine translation of sign language into spoken languages is an important yet non-trivial task. The sheer variety of dialects that exist in any sign language makes it only harder to come up with a generalized sign language classification system. Though a lot of work has been done in this area previously but most of the approaches rely on intrusive hardware in the form of wired or colored gloves or are specific language/dialect dependent for accurate sign language interpretation. We propose a cost-effective, non-intrusive webcam based solution in which a person from any part of the world can train our system to make it learn the sign language in their own specific dialect, so that our software can then correctly translate the hand signs into a commonly spoken language, such as English. Image based hand gesture recognition carries sheer importance in this task. The heart of hand gesture recognition systems is the detection and extraction of the sign (hand gesture) from the input image stream. Our work uses functions like skin color based thresholding, contour detection and convexity defect for detection of hands and identification of important points on the hand respectively. The distance of these important contour points from the centroid of the hand becomes our feature vector against which we train our neural network. The system works in two phases. In the training phase the correspondence between users hand gestures against each sign language symbol is learnt using a feed forward neural network with back propagation learning algorithm. Once the training is complete, user is free to use our system for translation or communication with other people. Experimental results based on training and testing the system with numerous users show that the proposed method can work well for dialectfree sign language translation (numerals and alphabets) and gives us average recognition accuracies of around 65% and 55% with the maximum recognition accuracies rising upto 77% and 62% respectively.

A Comprehensive Review of Sign Language Recognition: Different Types, Modalities, and Datasets

“…The feature extraction aid in accuracy improvement, and speed [47]. Some of these feature extraction method include SURF (Speeded Up Robust Feature) [34], speed up robust feature (Laplace of Gaussian with box filter) [34], SIFT (shift-invariant feature transform) [33], PCA (Principal Component Analysis) [37], [4], LDA (Linear Discriminant Analysis) [48], Convexity defects and k-curvature [49], time domain to frequency domain [31], [35], Local binary pattern, etc. The feature extraction methods used for SLR-based study is tabulated in Table III.…”

Implementation of image processing based Digital Dactylology Converser for deaf-mute persons

“…Statistic shows that out of the total population, 5.035 million are disabled where 0.38 million suffer from speaking loss or mute which is around 7.5% of the overall disabled population [1]. It is very important to understand that what an impaired hearing person feels and goes through.…”