Real time sign language recognition using the leap motion controller

Naglot, Deepali; Kulkarni, Milind

doi:10.1109/inventive.2016.7830097

Cited by 56 publications

(24 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly to the aforementioned work, researchers found that a different dataset also consisting of 26 ASL letters were classifiable at 93.81% accuracy with a Deep Neural Network [ 23 ]. Another example achieved 96.15% with a deep learning approach on a limited set of 520 samples (20 per letter) [ 24 ]. Data fusion via Coupled Hidden Markov Models was performed in [ 25 ] between Leap Motion and Kinect, which achieved 90.8% accuracy on a set of 25 Indian Sign Language gestures.…”

Section: Related Workmentioning

confidence: 99%

British Sign Language Recognition via Late Fusion of Computer Vision and Leap Motion with Transfer Learning to American Sign Language

Bird

Ekárt

Faria

2020

Sensors

View full text Add to dashboard Cite

In this work, we show that a late fusion approach to multimodality in sign language recognition improves the overall ability of the model in comparison to the singular approaches of image classification (88.14%) and Leap Motion data classification (72.73%). With a large synchronous dataset of 18 BSL gestures collected from multiple subjects, two deep neural networks are benchmarked and compared to derive a best topology for each. The Vision model is implemented by a Convolutional Neural Network and optimised Artificial Neural Network, and the Leap Motion model is implemented by an evolutionary search of Artificial Neural Network topology. Next, the two best networks are fused for synchronised processing, which results in a better overall result (94.44%) as complementary features are learnt in addition to the original task. The hypothesis is further supported by application of the three models to a set of completely unseen data where a multimodality approach achieves the best results relative to the single sensor method. When transfer learning with the weights trained via British Sign Language, all three models outperform standard random weight distribution when classifying American Sign Language (ASL), and the best model overall for ASL classification was the transfer learning multimodality approach, which scored 82.55% accuracy.

show abstract

Section: Related Workmentioning

confidence: 99%

British Sign Language Recognition via Late Fusion of Computer Vision and Leap Motion with Transfer Learning to American Sign Language

Bird

Ekárt

Faria

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…It is expected that sensors for the acquisition of hands skeletal data will be improved soon. Therefore, work is underway to use them to recognize sign languages: American [15][16][17][18][19][20][21][22][23][24], Arabic [25][26][27][28], Australian [13], Indian [29][30][31], Mexican [32], Pakistani [33], and Polish [34].…”

Section: Related Workmentioning

confidence: 99%

“…In [20], 26 letters of the American Finger Alphabet were recognized. The features measured by the LM controller and the Multilayer Perceptron (MLP) classifier were used.…”

Section: Related Workmentioning

confidence: 99%

Hand Posture Recognition Using Skeletal Data and Distance Descriptor

Kapuściński

Warchoł

2020

Applied Sciences

View full text Add to dashboard Cite

In this paper, a method for the recognition of static hand postures based on skeletal data was presented. A novel descriptor was proposed. It encodes information about distances between particular hand points. Five different classifiers were tested, including four common methods and a proposed modification of nearest neighbor classifier, which can distinguish between posture classes differing mostly in hand orientation. The experiments were performed using three challenging datasets of gestures from Polish and American Sign Languages. The proposed method was compared with other approaches found in the literature. It outperforms every compared method, including our previous work, in terms of recognition rate.

show abstract

“…Data is collected using leap motion controller, which is a USB peripheral device which is designed to allow users to control their computers using hand gestures alone [7]. This sensor can track hands, fingers, bones, and finger-like objects.…”

Section: Related Workmentioning

confidence: 99%

A Survey on Sign Language Detection

Purkayastha¹,

Jyoti²,

Pradhan³

et al. 2018

IJCA

View full text Add to dashboard Cite

The most important aspect of a society is communication. Every culture in the world has their own way of communicating with people of their kind. But the people who cannot hear or speak cannot easily share their thoughts. This creates a sort of isolation for people like them. The hand gestures that people of special abilities make to communicate would be taken up from a camera and the same would be translated and show on the screen along with an audible source. The most basic methods one has to follow in most of the sign language detection system are tracking, edge detection, segmentation, and a dataset. The first and foremost part is the recognition of hand or motion tracking of the hand from the camera. After the hand is successfully detected the extra environment must be diminished. The removal of an extra environment is done with the help of edge detection and segmentation through various image processing algorithms. After the distinguished picture is obtained, with the help of machine learning techniques and neural networks the result obtained are compared to the existing normalized dataset which is then translated over and produced as an output.

show abstract

Real time sign language recognition using the leap motion controller

Cited by 56 publications

References 10 publications

British Sign Language Recognition via Late Fusion of Computer Vision and Leap Motion with Transfer Learning to American Sign Language

British Sign Language Recognition via Late Fusion of Computer Vision and Leap Motion with Transfer Learning to American Sign Language

Hand Posture Recognition Using Skeletal Data and Distance Descriptor

A Survey on Sign Language Detection

Contact Info

Product

Resources

About