Gesture Recognition System for Isolated Word  Sign Language  Based on Key-Point Trajectory Matrix

Fakhfakh, Sana; Jemâa, Yousra Ben

doi:10.13053/cys-22-4-3046

Cited by 9 publications

(5 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The algorithm in document 32 combines the global features of AM-ResC3D and the local features of RCNN. The algorithm in reference 33 takes into account the key points and their trajectory characteristics. The algorithm in reference 34 is a framework based on Recurrent Neural Network (RNN).…”

Section: Experimental Results and Analysismentioning

confidence: 99%

“…Zhang et al 32 proposed a sign language recognition framework based on AM-ResC3D global feature analysis and Mask RCNN local feature description. Fakhfakh et al 33 proposed an algorithm to fuse dynamic and static features. The static level is the key point of the head/hand, and the dynamic level is the accumulation of the key point trajectory matrix.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improved 3D-ResNet sign language recognition algorithm with enhanced hand features

Wang

Yang

et al. 2022

Sci Rep

View full text Add to dashboard Cite

In sign language video, the hand region is small, the resolution is low, the motion speed is fast, and there are cross occlusion and blur phenomena, which have a great impact on sign language recognition rate and speed, and are important factors restricting sign language recognition performance. To solve these problems, this paper proposes an improved 3D-ResNet sign language recognition algorithm with enhanced hand features, aiming to highlight the features of both hands, solve the problem of missing more effective information when relying only on global features, and improve the accuracy of sign language recognition. The proposed method has two improvements. Firstly, the algorithm detects the left and right hand regions based on the improved EfficientDet network, uses the improved Bi-FPN module and dual channel and spatial attention module are used to enhance the detection ability of the network for small targets like hand. Secondly, the improved residual module is used to improve the 3D-ResNet18 network to extract sign language features. The global, the left-hand and the right-hand image sequences are divided into three branches for feature extraction and fusion, so as to strengthen the attention to hand features, strengthen the representation ability of sign language features, and achieve the purpose of improving the accuracy of sign language recognition. In order to verify the performance of this algorithm, a series of experiments are carried out on CSL dataset. For example, in the experiments of hand detection algorithm and sign language recognition algorithm, the performance indicators such as Top-N, mAP, FLOPs and Parm are applied to find the optimal algorithm framework. The experimental results show that the Top1 recognition accuracy of this algorithm reaches 91.12%, which is more than 10% higher than that of C3D, P3D and 3D-ResNet basic networks. From the performance indicators of Top-N, mAP, FLOPs, Parm and so on, the performance of the algorithm in this paper is better than several algorithms in recent three years, such as I3D+BLSTM, B3D ResNet, AM-ResC3D+RCNN and so on. The results show that the hand detection network with enhanced hand features and three-dimensional convolutional neural network proposed in this paper can achieve higher accuracy of sign language recognition.

show abstract

Section: Experimental Results and Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Improved 3D-ResNet sign language recognition algorithm with enhanced hand features

Wang

Yang

et al. 2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Vision-based approach is based only on standard cameras and rely on image-processing techniques to interpret gestures. The vision-based approach is more natural and easier to use than the sensor-based approach [13].…”

Section: Introductionmentioning

confidence: 99%

Real Time System Based Deep Learning for Recognizing Algerian Sign Language

Ahmed,

Imene,

El Bachir

2024

IJCDS

View full text Add to dashboard Cite

Sign language plays a crucial role in facilitating communication and interaction for the deaf community. However, the recognition of sign language poses unique challenges, especially in the context of Algerian Sign Language (ALGSL), where limited research has been conducted. Using recent advances in the field of deep learning, we present a novel ALGSL recoginition system using hand cropping and hand landmarks from successive video frames. Also, we propose a new key frame selection method to find a sufficient number of successive frames for the recognition decision, in order to cope with a near real-time system, where tradeoff between accuracy and response time is crucial to avoid delayed sign recognition. Our system is based on Autoencoder architecture enhanced by attention mechanism. The Autoencoder architecture combines both convolutional neural networks (CNN) for capturing spatial information and long-short-term memory (LSTM) for capturing temporal information. The proposed architecture is evaluated on our new ALGSL dataset and achieved an accuracy of 98,99%. Additionally, we test our architecture on different publicly datasets and shows outstanding results. Finally, we test the recognition of ALGSL gestures of our system for videos captured through a webcam.

show abstract

“…Sign language has different types: Isolated sign, which is one sign gesture; a continuous sign which offers a complete clause sign [5]. All sign language recognition systems contain four primary stages: dataset collecting, preprocessing, features extracting, and classifying signs images or videos.…”

Section: Introductionmentioning

confidence: 99%

Automatic Translation From Iraqi Sign Language to Arabic Text or Speech Using CNN

2023

IJCCCE

View full text Add to dashboard Cite

Sign language (SL) is Non-verbal communication and a way for the deaf and mute to communicate without words. A deaf and mute person's hands, face, and body shows what they want to say. Since the number of deaf and dumb people is increasing, there must be other ways to learn sign language or communicate with deaf and dumb people. One of these ways is using advanced technology to produce systems that help the deaf/dumb, such as creating recognition and sign language translators. This paper presents an application that works on the computer for machine translation of Iraqi sign language in two directions from sign language to Arabic language (text/speech) and from Arabic language(text) to Iraqi sign language. The proposed system uses a Convolution Neural Network (CNN) to classify sign language based on its features to predicate the sign meaning. The sign language to Arabic language(text/speech) part of the proposed system has an accuracy of 99.3% for letters. Index Terms—Sign Language, Iraqi Sign Language, Deep Learning, CNN, gtts.

show abstract

Gesture Recognition System for Isolated Word Sign Language Based on Key-Point Trajectory Matrix

Cited by 9 publications

References 32 publications

Improved 3D-ResNet sign language recognition algorithm with enhanced hand features

Improved 3D-ResNet sign language recognition algorithm with enhanced hand features

Real Time System Based Deep Learning for Recognizing Algerian Sign Language

Automatic Translation From Iraqi Sign Language to Arabic Text or Speech Using CNN

Contact Info

Product

Resources

About