Hand Gesture Recognition in Indian Sign Language Using Deep Learning

Vashisth, Harsh Kumar; Tarafder, Tuhin; Aziz, Rehan; Arora, Mamta; Alpana, Alpana

doi:10.3390/engproc2023059096

Cited by 4 publications

(1 citation statement)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vashisth et al [12] embarked on a significant endeavor to recognize Indian Sign Language using deep learning techniques, demonstrating the potential of neural networks in deciphering complex hand gestures. This study's importance lies in its focus on a less commonly researched sign language, offering valuable insights into the diversity and complexity of sign language recognition tasks.…”

Section: Related Workmentioning

confidence: 99%

Applying Swin Architecture to Diverse Sign Language Datasets

Kumar,

Huang,

Lin

et al. 2024

Electronics

View full text Add to dashboard Cite

In an era where artificial intelligence (AI) bridges crucial communication gaps, this study extends AI’s utility to American and Taiwan Sign Language (ASL and TSL) communities through advanced models like the hierarchical vision transformer with shifted windows (Swin). This research evaluates Swin’s adaptability across sign languages, aiming for a universal platform for the unvoiced. Utilizing deep learning and transformer technologies, it has developed prototypes for ASL-to-English translation, supported by an educational framework to facilitate learning and comprehension, with the intention to include more languages in the future. This study highlights the efficacy of the Swin model, along with other models such as the vision transformer with deformable attention (DAT), ResNet-50, and VGG-16, in ASL recognition. The Swin model’s accuracy across various datasets underscore its potential. Additionally, this research explores the challenges of balancing accuracy with the need for real-time, portable language recognition capabilities and introduces the use of cutting-edge transformer models like Swin, DAT, and video Swin transformers for diverse datasets in sign language recognition. This study explores the integration of multimodality and large language models (LLMs) to promote global inclusivity. Future efforts will focus on enhancing these models and expanding their linguistic reach, with an emphasis on real-time translation applications and educational frameworks. These achievements not only advance the technology of sign language recognition but also provide more effective communication tools for the deaf and hard-of-hearing community.

show abstract

Section: Related Workmentioning

confidence: 99%

Applying Swin Architecture to Diverse Sign Language Datasets

Kumar,

Huang,

Lin

et al. 2024

Electronics

View full text Add to dashboard Cite

show abstract

Using Human-Computer Interaction (HCI) and Artificial Intelligence (AI) in Education to Improve the Literacy of Deaf and Hearing-Impaired Children

Bekeš,

Galzina,

Kolar

2024

2024 47th MIPRO ICT and Electronics Convention (MIPRO)

View full text Add to dashboard Cite

Applying Swin Architecture to diverse Sign Language Datasets

Kumar,

Huang,

Lin

et al. 2024

Preprint

View full text Add to dashboard Cite

In the era of Artificial Intelligence (AI), comprehending and responding to non-verbal communication is increasingly vital. This research extends AI's reach in bridging communication gaps, notably benefiting American Sign Language (ASL) and Taiwan Sign Language (TSL) communities. It focuses on employing various AI models, especially the Hierarchical Vision Transformer with Shifted Windows (Swin), for recognizing diverse sign language datasets. The study assesses Swin architecture's adaptability to different sign languages, aiming to create a universal platform for Unvoiced communities. Utilizing deep learning and transformer technologies, hybrid application prototypes have been developed for ASL-to-English translations and vice versa, with plans to expand this to multiple sign languages. The Swin models, trained on varied dataset sizes, show considerable accuracy, indicating their flexibility and effectiveness. This research underscores major advancements in sign language recognition and underlines a commitment to inclusive communication in the digital era. Future work will focus on enhancing these models and broadening their scope to include more sign languages, integrating multimodality and Large Language Models (LLMs), thereby fostering global inclusivity.

show abstract

Hand Gesture Recognition in Indian Sign Language Using Deep Learning

Abstract: This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Cited by 4 publications

References 18 publications

Applying Swin Architecture to Diverse Sign Language Datasets

Applying Swin Architecture to Diverse Sign Language Datasets

Using Human-Computer Interaction (HCI) and Artificial Intelligence (AI) in Education to Improve the Literacy of Deaf and Hearing-Impaired Children

Applying Swin Architecture to diverse Sign Language Datasets

Contact Info

Product

Resources

About