Sign Language Recognition Analysis using Multimodal Data

Hosain, Al Amin; Santhalingam, Panneer Selvam; Pathak, Parth H.; Košecká, Jana; Rangwala, Huzefa

doi:10.1109/dsaa.2019.00035

Cited by 20 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, FineHand embedder learns representation from of fine grained hand shapes which has proven to be crucial for this kind of significant performance gain in classification. Our best method outperforms the work came with the dataset [8] by 12%.…”

Section: Resultsmentioning

confidence: 88%

“…All of our hand shape learning as well as ASL classification tasks were evaluated using GMU-ASL51 benchmark [8]. We picked this dataset because it is the only publicly available dataset of this type (isolated word level ASL gestures) with large number of sign variation.…”

Section: Gmu-asl51 Datasetmentioning

confidence: 99%

“…We will demonstrate superior performance of the proposed model on the GMU-ASL51 dataset [8] and compare it quantitatively with several baseline approaches which use different representations and different sensing modalities.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

FineHand: Learning Hand Shapes for American Sign Language Recognition

Hosain¹,

Santhalingam²,

Pathak³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

American Sign Language recognition is a difficult gesture recognition problem, characterized by fast, highly articulate gestures. These are comprised of arm movements with different hand shapes, facial expression and head movements. Among these components, hand shape is the vital, often the most discriminative part of a gesture. In this work, we present an approach for effective learning of hand shape embeddings, which are discriminative for ASL gestures. For hand shape recognition our method uses a mix of manually labelled hand shapes and high confidence predictions to train deep convolutional neural network (CNN). The sequential gesture component is captured by recursive neural network (RNN) trained on the embeddings learned in the first stage. We will demonstrate that higher quality hand shape models can significantly improve the accuracy of final video gesture classification in challenging conditions with variety of speakers, different illumination and significant motion blurr. We compare our model to alternative approaches exploiting different modalities and representations of the data and show improved video gesture recognition accuracy on GMU-ASL51 benchmark dataset.

show abstract

Section: Resultsmentioning

confidence: 88%

Section: Gmu-asl51 Datasetmentioning

confidence: 99%

See 1 more Smart Citation

FineHand: Learning Hand Shapes for American Sign Language Recognition

Hosain¹,

Santhalingam²,

Pathak³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is the challenge facing those responsible for higher education institutions the world over. In short, teaching staff need to be trained in how to design teaching activities that include SRL processes through ALT, since these resources allow the planning stage to be focused, for example with the use of avatars ( Azevedo et al, 2015 , Azevedo and Gašević, 2019 ), the follow-up stage ( Azevedo and Gašević, 2019 ; Hosain et al, 2019 ; Noroozi et al, 2019 ), for example using tools similar to UBUMonitor ( Sáiz-Manzanares et al, 2021b ), and the self-evaluation stage ( Kramarski and Michalsky, 2009 ; Bernardo et al, 2017 ), for example through the use of gamification activities so as to ultimately enhance motivation towards the goal of learning ( Zimmerman, 2008 ). All of this will aid student development of metacognitive learning strategies when processing information ( Carlson, 2003 ; Rothbart et al, 2011 ; McClelland et al, 2014 ; Valadas et al, 2017 ).…”

Section: Discussionmentioning

confidence: 99%

“…Moreover, the joint use of LMS and ALT enables interactions to be recorded ( Azevedo and Gašević, 2019 ; Hosain et al, 2019 ; Noroozi et al, 2019 ). The use thereof accounts for over 72% of variance in student learning outcomes ( Sáiz-Manzanares et al, 2019a ).…”

Section: Introductionmentioning

confidence: 99%

Teacher Training Effectiveness in Self-Regulation in Virtual Environments

et al. 2022

View full text Add to dashboard Cite

Higher education in the 21st century faces the challenge of changing the way in which knowledge is conveyed and how teachers and students interact in the teaching-learning process. The current pandemic caused by SARS-CoV-2 has hastened the need to face up to this challenge and has furthered the need to approach the issue from the perspective of digitalisation. To achieve this, it is necessary to design training programmes geared towards teaching staff and which address both the use of technology and instructional design aimed at promoting the development of self-regulated learning (SRL) and automatic feedback systems. In this study, work was carried out with 23 teachers (8 inexperienced and 15 experienced teachers) in a training programme conducted through Moodle. The aims were: (1) to test whether there were any significant differences between the behaviour patterns of new teachers compared to experienced teachers, (2) to determine whether clusters of behaviour patterns corresponded to the type of teacher and (3) to ascertain whether the level of teacher satisfaction with the training activity in digital teaching will depend on the type of teacher. A quantitative as well as a qualitative design was applied. Differences were found in the behaviour patterns in the training activities for the development of rubrics and use of learning analytics systems in virtual learning environments. It was also found that the type of teacher did not correspond exactly to the behaviour cluster in the learning platform. In addition, no significant differences were found in the level of satisfaction between the two kinds of teacher. The main contribution this study makes is to provide a detailed description of the training stage as well as the materials required for its repetition. Further analytical studies are required on teacher perception of training programmes in digital teaching in order to provide personalised training proposals that lead to an effective use of teaching in digital environments.

show abstract