An AI-based Approach for Improved Sign Language Recognition using Multiple Videos

Dignan, Cameron; Perez, Eliud; Ahmad, Ishfaq; Huber, Manfred; Clark, Addison

doi:10.1007/s11042-021-11830-y

Cited by 6 publications

(1 citation statement)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, researchers are constantly developing new techniques and algorithms to improve the accuracy and robustness of HGR systems. In line with the trend of learning systems, convolutional neural networks (CNNs) have been successfully applied in image recognition tasks, while recurrent neural networks (RNNs) are a natural choice in recognizing gestures from videos [ 14 , 15 ]. One solution for HGR is to include a hand segmentation module as the first stage of the pipeline [ 16 , 17 ].…”

Section: Introductionmentioning

confidence: 99%

Domain Adaptation with Contrastive Simultaneous Multi-Loss Training for Hand Gesture Recognition

Baptista

Santos

Silva

et al. 2023

Sensors

View full text Add to dashboard Cite

Hand gesture recognition from images is a critical task with various real-world applications, particularly in the field of human–robot interaction. Industrial environments, where non-verbal communication is preferred, are significant areas of application for gesture recognition. However, these environments are often unstructured and noisy, with complex and dynamic backgrounds, making accurate hand segmentation a challenging task. Currently, most solutions employ heavy preprocessing to segment the hand, followed by the application of deep learning models to classify the gestures. To address this challenge and develop a more robust and generalizable classification model, we propose a new form of domain adaptation using multi-loss training and contrastive learning. Our approach is particularly relevant in industrial collaborative scenarios, where hand segmentation is difficult and context-dependent. In this paper, we present an innovative solution that further challenges the existing approach by testing the model on an entirely unrelated dataset with different users. We use a dataset for training and validation and demonstrate that contrastive learning techniques in simultaneous multi-loss functions provide superior performance in hand gesture recognition compared to conventional approaches in similar conditions.

show abstract