Bangla Sign alphabet recognition with zero-shot and transfer learning

Nihal, Ragib Amin; Rahman, Sejuti; Broti, Nawara Mahmood; Deowan, Shamim Ahmed

doi:10.1016/j.patrec.2021.06.020

Cited by 25 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sign language vocabulary is so large that it is difficult to capture it all by building a dataset, so getting the models to learn the new sign language themselves will be one of the main solutions. Currently, some scholars [164][165][166] have carried out research on isolated sign language recognition based on fewshot learning or zero-shot learning. There is still a large room for improvement.…”

Section: No Ability To Recognize Sign Language Outside the Datasetmentioning

confidence: 99%

Sign Language Recognition: A Comprehensive Review of Traditional and Deep Learning Approaches, Datasets, and Challenges

Tao,

Zhao,

Liu

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The Deaf are a large social group in society. Their unique way of communicating through sign language is often confined within their community due to limited understanding by individuals outside of this demographic. This is where sign language recognition (SLR) comes in to help normal people understand the meaning of sign language. In recent years, new methods of sign language recognition have been developed and achieved good results, so it is necessary to make a summary. This review mainly focuses on the introduction of sign language recognition techniques based on algorithms especially in recent years, including the recognition models based on traditional methods and deep learning approaches, sign language datasets, challenges and future directions in SLR. To make the method structure clearer, this article explains and compares the basic principles of different methods from the perspectives of feature extraction and temporal modelling. We hope that this review will provide some reference and help for future research in sign language recognition.

show abstract

Section: No Ability To Recognize Sign Language Outside the Datasetmentioning

confidence: 99%

Sign Language Recognition: A Comprehensive Review of Traditional and Deep Learning Approaches, Datasets, and Challenges

Tao,

Zhao,

Liu

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…This model is trained to master conditional probability distribution; P(y|x, T). This learning type is suitable for machine translation, where the label is absent in the target language [244][245][246].…”

Section: Zero-shot Learningmentioning

confidence: 99%

A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications

et al. 2023

View full text Add to dashboard Cite

Data scarcity is a major challenge when training deep learning (DL) models. DL demands a large amount of data to achieve exceptional performance. Unfortunately, many applications have small or inadequate data to train DL frameworks. Usually, manual labeling is needed to provide labeled data, which typically involves human annotators with a vast background of knowledge. This annotation process is costly, time-consuming, and error-prone. Usually, every DL framework is fed by a significant amount of labeled data to automatically learn representations. Ultimately, a larger amount of data would generate a better DL model and its performance is also application dependent. This issue is the main barrier for many applications dismissing the use of DL. Having sufficient data is the first step toward any successful and trustworthy DL application. This paper presents a holistic survey on state-of-the-art techniques to deal with training DL models to overcome three challenges including small, imbalanced datasets, and lack of generalization. This survey starts by listing the learning techniques. Next, the types of DL architectures are introduced. After that, state-of-the-art solutions to address the issue of lack of training data are listed, such as Transfer Learning (TL), Self-Supervised Learning (SSL), Generative Adversarial Networks (GANs), Model Architecture (MA), Physics-Informed Neural Network (PINN), and Deep Synthetic Minority Oversampling Technique (DeepSMOTE). Then, these solutions were followed by some related tips about data acquisition needed prior to training purposes, as well as recommendations for ensuring the trustworthiness of the training dataset. The survey ends with a list of applications that suffer from data scarcity, several alternatives are proposed in order to generate more data in each application including Electromagnetic Imaging (EMI), Civil Structural Health Monitoring, Medical imaging, Meteorology, Wireless Communications, Fluid Mechanics, Microelectromechanical system, and Cybersecurity. To the best of the authors’ knowledge, this is the first review that offers a comprehensive overview on strategies to tackle data scarcity in DL.

show abstract

“…The representations learned from the large-scale datasets facilitate learning from the small-scale datasets. Nihal et al [11] observed that computervision tasks share similar features. They trained a model on ImageNet and transferred the knowledge to Bangla sign alphabet recognition [12].…”

Section: Figurementioning

confidence: 99%

RFaNet: Receptive Field-Aware Network with Finger Attention for Fingerspelling Recognition Using a Depth Sensor

et al. 2021

View full text Add to dashboard Cite

Automatic fingerspelling recognition tackles the communication barrier between deaf and hearing individuals. However, the accuracy of fingerspelling recognition is reduced by high intra-class variability and low inter-class variability. In the existing methods, regular convolutional kernels, which have limited receptive fields (RFs) and often cannot detect subtle discriminative details, are applied to learn features. In this study, we propose a receptive field-aware network with finger attention (RFaNet) that highlights the finger regions and builds inter-finger relations. To highlight the discriminative details of these fingers, RFaNet reweights the low-level features of the hand depth image with those of the non-forearm image and improves finger localization, even when the wrist is occluded. RFaNet captures neighboring and inter-region dependencies between fingers in high-level features. An atrous convolution procedure enlarges the RFs at multiple scales and a non-local operation computes the interactions between multi-scale feature maps, thereby facilitating the building of inter-finger relations. Thus, the representation of a sign is invariant to viewpoint changes, which are primarily responsible for intra-class variability. On an American Sign Language fingerspelling dataset, RFaNet achieved 1.77% higher classification accuracy than state-of-the-art methods. RFaNet achieved effective transfer learning when the number of labeled depth images was insufficient. The fingerspelling representation of a depth image can be effectively transferred from large- to small-scale datasets via highlighting the finger regions and building inter-finger relations, thereby reducing the requirement for expensive fingerspelling annotations.

show abstract

Bangla Sign alphabet recognition with zero-shot and transfer learning

Cited by 25 publications

References 9 publications

Sign Language Recognition: A Comprehensive Review of Traditional and Deep Learning Approaches, Datasets, and Challenges

Sign Language Recognition: A Comprehensive Review of Traditional and Deep Learning Approaches, Datasets, and Challenges

A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications

RFaNet: Receptive Field-Aware Network with Finger Attention for Fingerspelling Recognition Using a Depth Sensor

Contact Info

Product

Resources

About