Real-time Bangla Sign Language Detection using Xception Model with Augmented Dataset

Urmee, Progya Paromita; Mashud, Md. Abdullah Al; Akter, Jasmin; Jameel, Abu Shafin Mohammad Mahdee; Islam, Salekul

doi:10.1109/wiecon-ece48653.2019.9019934

Cited by 22 publications

(5 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The available BdSL datasets are not sufficient enough [11,12,17,18,19] to develop a fully functioned real-time BdSL detection system. Half of these datasets [11,12,17,20] are also not available for further research. These datasets are further built on a controlled lab environment and the only exception is [18] but this dataset only contains 10 classes.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

BdSL36: A Dataset for Bangladeshi Sign Letters Recognition

Hoque¹,

Jubair²,

Akash³

et al. 2021

Computer Vision – ACCV 2020 Workshops

View full text Add to dashboard Cite

Bangladeshi Sign Language (BdSL) is a commonly used medium of communication for the hearing-impaired people in Bangladesh. A real-time BdSL interpreter with no controlled lab environment has a broad social impact and an interesting avenue of research as well. Also, it is a challenging task due to the variation in different subjects (age, gender, color, etc.), complex features, and similarities of signs and clustered backgrounds. However, the existing dataset for BdSL classification task is mainly built in a lab friendly setup which limits the application of powerful deep learning technology. In this paper, we introduce a dataset named BdSL36 which incorporates background augmentation to make the dataset versatile and contains over four million images belonging to 36 categories. Besides, we annotate about 40, 000 images with bounding boxes to utilize the potentiality of object detection algorithms. Furthermore, several intensive experiments are performed to establish the baseline performance of our BdSL36. Moreover, we employ beta testing of our classifiers at the user level to justify the possibilities of real-world application with this dataset. We believe our BdSL36 will expedite future research on practical sign letter classification. We make the datasets and all the pre-trained models available for further researcher.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In Table 1, we compare the BdSL36 with other existing datasets related to the task of BdSL letter recognition. Most of the available BdSL [12,18,19,20,28,29,3] Fig. 3: The top left image is a raw image captured by one of the volunteers.…”

Section: Comparison With Other Datasetsmentioning

confidence: 99%

BdSL36: A Dataset for Bangladeshi Sign Letters Recognition

Hoque¹,

Jubair²,

Akash³

et al. 2021

Computer Vision – ACCV 2020 Workshops

View full text Add to dashboard Cite

show abstract

“…They proposed a network of VGG19 architecture to recognize static hand gestures with an accuracy of 89.6%. Similarly, Urmee et al [30] employed the Xception architecture on a dataset of 2000 images with 37 different signs obtaining an accuracy of 98.93%. Yasir et al used a virtual reality-based hand tracking controller to track the hand motion using CNN-based architecture.…”

Section: B Deep Learning Techniquesmentioning

confidence: 99%

Generalization of Bangla Sign Language Recognition Using Angular Loss Functions

Youme¹,

Chowdhury

Ahamed

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Sign Language provides the means of conveying messages for deaf and mute people. Effective communication with the masses is a great challenge for the deaf and mute community, as Sign Language is not commonly understood. Many researchers have done numerous works in foreign language datasets like English, French, Japanese, etc. However, for Bangla, one of the most widely spoken languages, much significant work has not been done yet. Most of the works on Bangla Sign Language are executed on small datasets and report satisfactory performance. However, when small datasets are evaluated from the perspective of generalizability, particularly when using deep learning based solutions, these models fail to reproduce to the same performance. Therefore, this paper poses inter-dataset evaluation as the main evaluation criteria and evaluates several deep learning based models. This evaluation is done for Bangla by leveraging two popular datasets of Bangla Sign Language. Unsurprisingly, the inter-dataset performance is inferior, and several approaches to improve are explored and documented, including the use of angular margin based loss functions. The results demonstrate the importance of such an evaluation and also show that one of the proposed approaches shows promising performance, albeit with significant room for improvement. This raises the need for a standardized dataset to overcome this issue of generalization for real-life applications, as well as the need to encourage future works to concentrate on challenging evaluations instead of pursuing deceptively good intra-dataset performance.

show abstract

“…Numerous studies have been conducted on BdSL recognition, and there are numerous benchmarking datasets [3], [4], [5], [6], [7] for BdSL recognition. However, these datasets are insufficient for training and evaluating deep learning models, and the majority are not open-source.…”

Section: Introductionmentioning

confidence: 99%

QMX-BdSL49: An Efficient Recognition Approach for Bengali Sign Language with Quantize Modified Xception

Begum¹,

Khan²,

Rahman³

et al. 2023

IJACSA

View full text Add to dashboard Cite

Sign language is developed to bridge the communication gap between individuals with and without hearing impairment or speech difficulties. Individuals with hearing and speech impairment typically rely on hand signs as a means of expressing themselves. However, people, in general, may not have sufficient knowledge of sign language, thus a sign language recognition system on an embedded device is most needed. Literature related to such systems on embedded devices is scarce as these recognition tasks are very complex and computationally expensive. The limited resources of embedded devices cannot execute complex algorithms like Convolutional Neural Network (CNN) properly. Therefore, in this paper, we propose a novel deep learning architecture based on default Xception architecture, named Quantized Modified Xception (QMX) to reduce the model's size and enhance the computational speed without compromising model accuracy. Moreover, the proposed QMX model is highly optimized due to the weight compression of model quantization. As a result, the footprint of the proposed QMX model is 11 times smaller than the Modified Xception (MX) model. To train the model, BDSL 49 dataset is utilized which includes approximately 14,700 images divided into 49 classes. The proposed QMX model achieves an overall F1 accuracy of 98%. In addition, a comprehensive analysis among QMX, Modified Xception Tiny (MXT), MX, and the default Xception model is provided in this research. Finally, the model has been implemented on Raspberry Pi 4 and a detailed evaluation of its performance has been conducted, including a comparison with existing state-of-the-art approaches in this domain. The results demonstrate that the proposed QMX model outperforms the prior work in terms of performance.

show abstract

Real-time Bangla Sign Language Detection using Xception Model with Augmented Dataset

Cited by 22 publications

References 12 publications

BdSL36: A Dataset for Bangladeshi Sign Letters Recognition

BdSL36: A Dataset for Bangladeshi Sign Letters Recognition

Generalization of Bangla Sign Language Recognition Using Angular Loss Functions

QMX-BdSL49: An Efficient Recognition Approach for Bengali Sign Language with Quantize Modified Xception

Contact Info

Product

Resources

About