American Sign Language Character Recognition Using Convolution Neural Network

Masood, Sarfaraz; Thuwal, Harish Chandra; Srivastava, Adhyan

doi:10.1007/978-981-10-5547-8_42

Cited by 27 publications

(10 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatively, vision-based systems don't rely on such sensors, rather rely on the inputs given by the camera. Such systems mostly use convolutional neural networks for classifying hand gestures [4][5][6][7]. Pardasani et al used a convolutional neural network that is similar to LeNet5 [9] for classifying hand gestures [6].…”

Section: Related Workmentioning

confidence: 99%

“…However, the performance of this classifier is not up to the mark, attributing to the fact that it has very few layers. Masood et al used VGG16 [10] as the classifier for sign language [5]. While VGG16 is a very accurate architecture and its performance is appreciable, the main drawback of VGG16 is that it is gigantic, with over 100 million parameters which aren't amicable to be run on an embedded platform as well as fail for real-time performance.…”

Section: Related Workmentioning

confidence: 99%

“…There are primarily two different methods for recognising hand gestures, the first one relies on sensor data from gloves such as flex sensor [2,3] and the other relies on vision-based methods that rely on the data captured from the camera and processes it to extract valuable information [4][5][6][7]. Vision-based methods are more popular and they mostly use Convolutional Neural Networks as they are known to have excellent classification abilities and are often used for this purpose.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Real-Time Sign Language Interpreter on Embedded Platform

Mulchandani

Paunwala

2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

Sign language is the primary way of communication for speechimpaired people. American Sign Language is one of the most popular sign languages and is used by over 250,000 people around the world. However, this language is only decipherable by the people who have the same disabilities, and it is not understood by masses. Hence, a need arises for an interpreter that can convert this language to text and speech. The existing research focuses on effectively converting sign language to text and speech; however, less emphasis is laid on building a system that is not only accurate but efficient as well which could be run on a compact embedded platform. This is necessary so that a portable interpreter can be made which can be easily carried and used by such people for their convenience making their day to day life easier. In this research, we propose an implementation based on MobileNet architecture, that is not only accurate but efficient as well. Moreover; a demonstration of the entire framework has also been shown on Jetson Nano.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Sign Language Interpreter on Embedded Platform

Mulchandani

Paunwala

2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

show abstract

“…In terms of RGB classification specifically, many state-of-the-art works have argued in favour of the VGG16 architecture [ 13 ] for hand gesture recognition towards sign language classification [ 14 ]. These works include British [ 15 ], American [ 16 ], Brazilian [ 17 ] and Bengali [ 18 ] Sign Languages, among others. Given the computational complexity of multimodality when visual methods are concerned in part, multimodality is a growing approach to hand gesture recognition.…”

Section: Related Workmentioning

confidence: 99%

British Sign Language Recognition via Late Fusion of Computer Vision and Leap Motion with Transfer Learning to American Sign Language

Bird

Ekárt

Faria

2020

Sensors

View full text Add to dashboard Cite

In this work, we show that a late fusion approach to multimodality in sign language recognition improves the overall ability of the model in comparison to the singular approaches of image classification (88.14%) and Leap Motion data classification (72.73%). With a large synchronous dataset of 18 BSL gestures collected from multiple subjects, two deep neural networks are benchmarked and compared to derive a best topology for each. The Vision model is implemented by a Convolutional Neural Network and optimised Artificial Neural Network, and the Leap Motion model is implemented by an evolutionary search of Artificial Neural Network topology. Next, the two best networks are fused for synchronised processing, which results in a better overall result (94.44%) as complementary features are learnt in addition to the original task. The hypothesis is further supported by application of the three models to a set of completely unseen data where a multimodality approach achieves the best results relative to the single sensor method. When transfer learning with the weights trained via British Sign Language, all three models outperform standard random weight distribution when classifying American Sign Language (ASL), and the best model overall for ASL classification was the transfer learning multimodality approach, which scored 82.55% accuracy.

show abstract

“…Using these, single word unigram probabilities and a handful of other heuristics, it re-turns the optimal word to the user. Masood et al (2018) in their paper American Sign Language Character Recognition using Convolution Neural Network proposes a vision-based system to identify symbols of ASL which includes Alphabets from A to Z and numbers ranging from 0 to 9. The first step of this approach is Image Augmentation and Resizing.…”

Section: C) Feature Extraction and Recognitionmentioning

confidence: 99%

A Multi-Modular Approach for Gesture Recognition and Text Formulation in American Sign Language

Joglekar¹,

Sawant²,

Jain³

et al. 2020

IJCATR

View full text Add to dashboard Cite

The most important thing in communication is hearing what isn't being said, this goes for the people who are deaf and mute. Sign language is the most natural means of exchanging information amongst these people. The deaf and mute people have to rely on an interpreter or some sort of visual communication while communicating with the rest of society. So, to bridge the gap between the deaf and dumb community and the rest of the world, there is the need for the translation system that will make the entire process of communication fluent. American Sign Language (ASL) is a visual and gestural language i.e. the brain processes linguistic information through eyes. This paper proposes a system that interprets the gestures in sign language enacted by the person to plain text. It is a robust as well as real-time Intelligent Translation System ensuring that none of these two parameters are compromised. This application would capture a live video stream of the user performing ASL from the webcam which then translates the gestures to corresponding text using Image Processing techniques and Deep Learning Models. It is then combined to form words that are passed to the spell checker and then combined to form sentences. The proposed solutions would thus enable quick and effective communication between the deaf and mute community and the rest of the society.

show abstract

American Sign Language Character Recognition Using Convolution Neural Network

Cited by 27 publications

References 9 publications

Real-Time Sign Language Interpreter on Embedded Platform

Real-Time Sign Language Interpreter on Embedded Platform

British Sign Language Recognition via Late Fusion of Computer Vision and Leap Motion with Transfer Learning to American Sign Language

A Multi-Modular Approach for Gesture Recognition and Text Formulation in American Sign Language

Contact Info

Product

Resources

About