Technical Approaches to Chinese Sign Language Processing: A Review

Kamal, Suhail Muhammad; Chen, Yidong; Li, Shaozi; Shi, Xiaodong; Zheng, Jiangbin

doi:10.1109/access.2019.2929174

Cited by 41 publications

(22 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other experiments have also shown interest in a variation of PCA, referred to as recursive principal component analysis (RPCA) for feature extraction. While exploring the features of SLR systems, [97] reported that using RPCA achieved a classification rate of 98%.…”

Section: Principal Component Analysismentioning

confidence: 99%

Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues

Al‐Qurishi¹,

Khalid²,

Souissi³

2021

IEEE Access

View full text Add to dashboard Cite

People with hearing impairments are found worldwide; therefore, the development of effective local level sign language recognition (SLR) tools is essential. We conducted a comprehensive review of automated sign language recognition based on machine/deep learning methods and techniques published between 2014 and 2021 and concluded that the current methods require conceptual classification to interpret all available data correctly. Thus, we turned our attention to elements that are common to almost all sign language recognition methodologies. This paper discusses their relative strengths and weaknesses, and we propose a general framework for researchers. This study also indicates that input modalities bear great significance in this field; it appears that recognition based on a combination of data sources, including vision-based and sensor-based channels, is superior to a unimodal analysis. In addition, recent advances have allowed researchers to move from simple recognition of sign language characters and words towards the capacity to translate continuous sign language communication with minimal delay. Many of the presented models are relatively effective for a range of tasks, but none currently possess the necessary generalization potential for commercial deployment. However, the pace of research is encouraging, and further progress is expected if specific difficulties are resolved.

show abstract

Section: Principal Component Analysismentioning

confidence: 99%

Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues

Al‐Qurishi¹,

Khalid²,

Souissi³

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Plenty of research works have developed to solve various sign language recognition tasks for different languages [14], [16]- [18]. The Chinese sign language recognition systems was reviewed in [17]. Xiao et al [19] utilized dual Long Short-Term Memory (LSTM) and a Couple Hidden Markov Model (CHMM) to fuse hand and skeleton sequence information.…”

Section: Related Workmentioning

confidence: 99%

DeepArSLR: A Novel Signer-Independent Deep Learning Framework for Isolated Arabic Sign Language Gestures Recognition

Aly

2020

IEEE Access

110

View full text Add to dashboard Cite

Hand gesture recognition has attracted the attention of many researchers due to its wide applications in robotics, games, virtual reality, sign language and human-computer interaction. Sign language is a structured form of hand gestures and the most effective communication way among hear-impaired people. Developing an efficient sign language recognition system to recognize dynamic isolated gestures encounters three major challenges, namely, hand segmentation, hand shape feature representation and gesture sequence recognition. Traditional sign language recognition methods utilize color-based hand segmentation algorithms to segment hands, hand-crafted feature extraction for hand shape representation and Hidden Markov Model (HMM) for sequence recognition. In this paper, a novel framework is proposed for signerindependent sign language recognition using multiple deep learning architectures comprising hand semantic segmentation, hand shape feature representation and deep recurrent neural network. The recently developed semantic segmentation method called DeepLabv3+ is trained using a set of pixel-labeled hand images to extract hand regions from each frame of the input video. Then, the extracted hand regions are cropped and scaled to a fixed size to alleviate hand scale variations. Extracting hand shape features is achieved using a single layer Convolutional Self-Organizing Map (CSOM) instead of relying on transfer learning of pretrained deep convolutional neural networks. The sequence of extracted feature vectors are then recognized using deep Bi-directional Long Short-Term Memory (BiLSTM) recurrent neural network. BiLSTM network contains three BiLSTM layers, one fully connected and softmax layers. The performance of the proposed method is evaluated using a challenging Arabic sign language database containing 23 isolated words captured from three different users. Experimental results show that the performance of proposed framework outperforms with large margin the state-of-the-art methods for signer-independent testing strategy. INDEX TERMS Arabic sign language recognition, deep learning, hand semantic segmentation, convolutional self-organizing map, signer-independent, deep BiLSTM network.

show abstract

“…According to a recent review [ 28 ], sign language is an ongoing research that began decades ago. The SLR system can be classified into three based on the type: (1) fingerspelling recognition; (2) isolated word recognition; (3) continuous sign sentence recognition.…”

Section: Related Workmentioning

confidence: 99%

An Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences

Zheng

Chen

et al. 2020

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

Sign language translation (SLT) is an important application to bridge the communication gap between deaf and hearing people. In recent years, the research on the SLT based on neural translation frameworks has attracted wide attention. Despite the progress, current SLT research is still in the initial stage. In fact, current systems perform poorly in processing long sign sentences, which often involve long-distance dependencies and require large resource consumption. To tackle this problem, we propose two explainable adaptations to the traditional neural SLT models using optimized tokenization-related modules. First, we introduce a frame stream density compression (FSDC) algorithm for detecting and reducing the redundant similar frames, which effectively shortens the long sign sentences without losing information. Then, we replace the traditional encoder in a neural machine translation (NMT) module with an improved architecture, which incorporates a temporal convolution (T-Conv) unit and a dynamic hierarchical bidirectional GRU (DH-BiGRU) unit sequentially. The improved component takes the temporal tokenization information into consideration to extract deeper information with reasonable resource consumption. Our experiments on the RWTH-PHOENIX-Weather 2014T dataset show that the proposed model outperforms the state-of-the-art baseline up to about 1.5+ BLEU-4 score gains.

show abstract

Technical Approaches to Chinese Sign Language Processing: A Review

Cited by 41 publications

References 43 publications

Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues

Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues

DeepArSLR: A Novel Signer-Independent Deep Learning Framework for Isolated Arabic Sign Language Gestures Recognition

An Improved Sign Language Translation Model with Explainable Adaptations for Processing Long Sign Sentences

Contact Info

Product

Resources

About