Real-Time Sign Language Recognition in Complex Background Scene Based on a Hierarchical Clustering Classification Method

Pan, Tse-Yu; Lo, Li-Yun; Yeh, Chung-Wei; Li, Jhe-Wei; Liu, Hou-Tim; Hu, Min‐Chun

doi:10.1109/bigmm.2016.44

Cited by 33 publications

(16 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In summary, the essence of the SIFT algorithms is keypoint detection and descriptor generation. References [16][17][18] employed SIFT to extract features. Figure 5 shows the representation of image matching process based on SIFT algorithm.…”

Section: Scale-invariant Feature Transformmentioning

confidence: 99%

“…As lower eigenvalues with less information, the data dimensionality is reduced and eigenvectors are stored effectively. Therefore, PCA features are widely applied [18,44,45] to improve the overall accuracy. Paper [15] proposed a novel hierarchical classification scheme integrated principal component analysis (PCA), linear discriminant analysis (LDA), and support vector machine (SVM), which achieved a higher accuracy.…”

Section: Principal Component Analysismentioning

confidence: 99%

“…Techniques used for feature extraction [16][17][18] Scale-invariant feature transform (SIFT) [20][21][22][23][24][25] Histogram of oriented gradients (HOG) [32][33][34][35] Dynamic time warping (DTW) [28] Wavelet entropy (WE) [36] Hu moment invariant (HMI) [37,39] Fourier descriptor (FD) [38,40] Gray-level co-occurrence matrix (GLCM) [41] Speeded up robust feature (SURF) [42] Latent Dirichlet allocation (LDA) Fig. 8 A toy example of dimensionality reduction of PCA reconstruction, prediction, and image super-resolution [46,47].…”

Section: Referencesmentioning

confidence: 99%

“…There may be two reasons to explain this phenomenon. One is that this type of sign language is closer to the English series, and its expression is relatively [16,18], FD, GMM [18], EFD [143], and HSV [44], which is also very beneficial to improve the overall accuracy.…”

Section: Characteristics Of Chinese Sign Language Recognitionmentioning

confidence: 99%

See 3 more Smart Citations

A Survey on Artificial Intelligence in Chinese Sign Language Recognition

et al. 2020

View full text Add to dashboard Cite

Chinese Sign Language (CSL) offers the main means of communication for the hearing impaired in China. Sign Language Recognition (SLR) can shorten the distance between the hearing-impaired and healthy people and help them integrate into the society. Therefore, SLR has become the focus of sign language application research. Over the years, the continuous development of new technologies provides a source and motivation for SLR. This paper aims to cover the most recent approaches in Chinese Sign Language Recognition (CSLR). With a thorough review of superior methods from 2000 to 2019 in CSLR researches, various techniques and algorithms such as scale-invariant feature transform, histogram of oriented gradients, wavelet entropy, Hu moment invariant, Fourier descriptor, gray-level co-occurrence matrix, dynamic time warping, principal component analysis, autoencoder, hidden Markov model (HMM), support vector machine (SVM), random forest, skin color modeling method, k-NN, artificial neural network, convolutional neural network (CNN), and transfer learning are discussed in detail, which are based on several major stages, that is, data acquisition, preprocessing, feature extraction, and classification. CSLR was summarized from some aspect as follows: methods of classification and feature extraction, accuracy/performance evaluation, and sample size/datasets. The advantages and limitations of different CSLR approaches were compared. It was found that data acquisition is mainly through Kinect and camera, and the feature extraction focuses on hand's shape and spatiotemporal factors, but ignoring facial expressions. HMM and SVM are used most in the classification. CNN is becoming more and more popular, and a deep neural network-based recognition approach will be the future trend. However, due to the complexity of the contemporary Chinese language, CSLR generally has a lower accuracy than other SLR. It is necessary to establish an appropriate dataset to conduct comparable experiments. The issue of decreasing accuracy as the dataset increases needs to resolve. Overall, our study is hoped to give a comprehensive presentation for those people who are interested in CSLR and SLR and to further contribute to the future research.

show abstract

Section: Scale-invariant Feature Transformmentioning

confidence: 99%

Section: Principal Component Analysismentioning

confidence: 99%

Section: Referencesmentioning

confidence: 99%

Section: Characteristics Of Chinese Sign Language Recognitionmentioning

confidence: 99%

See 2 more Smart Citations

A Survey on Artificial Intelligence in Chinese Sign Language Recognition

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The study captured images for each sign and applied background subtraction techniques to remove the background. Meanwhile, instead of background subtraction, Pan et al [ 11 ] presented hand segmentation by extracting skin color in the YCbCr color space determined by applying a Gaussian mixture model (GMM) to find the largest skin blob. YCbCr is a family of color spaces used in video and digital photography systems, where Y, Cb, and Cr represent the luma, blue, and red components, respectively.…”

Section: Introductionmentioning

confidence: 99%

American Sign Language Recognition Using Leap Motion Controller with Machine Learning Approach

Chong

Lee

2018

Sensors

109

View full text Add to dashboard Cite

Sign language is intentionally designed to allow deaf and dumb communities to convey messages and to connect with society. Unfortunately, learning and practicing sign language is not common among society; hence, this study developed a sign language recognition prototype using the Leap Motion Controller (LMC). Many existing studies have proposed methods for incomplete sign language recognition, whereas this study aimed for full American Sign Language (ASL) recognition, which consists of 26 letters and 10 digits. Most of the ASL letters are static (no movement), but certain ASL letters are dynamic (they require certain movements). Thus, this study also aimed to extract features from finger and hand motions to differentiate between the static and dynamic gestures. The experimental results revealed that the sign language recognition rates for the 26 letters using a support vector machine (SVM) and a deep neural network (DNN) are 80.30% and 93.81%, respectively. Meanwhile, the recognition rates for a combination of 26 letters and 10 digits are slightly lower, approximately 72.79% for the SVM and 88.79% for the DNN. As a result, the sign language recognition system has great potential for reducing the gap between deaf and dumb communities and others. The proposed prototype could also serve as an interpreter for the deaf and dumb in everyday life in service sectors, such as at the bank or post office.

show abstract