User-Defined Hand Gesture Interface to Improve User Experience of Learning American Sign Language

Abstract

Sign language can make possible effective communication between hearing and deaf-mute people. Despite years of extensive pedagogical research, learning sign language remains a formidable task, with the majority of the current systems relying extensively on online learning resources, presuming that users would regularly access them; yet, this approach can feel monotonous and repetitious. Recently, gamification has been proposed as a solution to the problem, however, the research focus is on game design, rather than user experience design. In this work, we present a system for user-defined interaction for learning static American Sign Language (ASL), supporting gesture recognition for user experience design, and enabling users to actively learn through involvement with user-defined gestures, rather than just passively absorbing knowledge. Early findings from a questionnaire-based survey show that users are more motivated to learn static ASL through user-defined interactions.

LBKT: A LSTM BERT-Based Knowledge Tracing Model for Long-Sequence Data

“…It aims to personalize feedback and instruction, enhancing learning outcomes. KT methods are categorized into probabilistic, logistic, and deep learning-based models [6,31,29]. Probabilistic models, like Bayesian Knowledge Tracing (BKT), utilize Hidden Markov Models or Bayesian Belief Networks to track learning states, but struggle with complexity and multi-skill scenarios [6,27,10,30].…”

Comparative Efficacy of 2D and 3D Virtual Reality Games in American Sign Language Learning

“…In our study, we developed 2D and 3D versions of a whack-amole game that facilitates learning the American Sign Language (ASL) numbers from 0 to 9 and conducted a user study to compare the two versions. The interface has common elements with prior work by the authors presented in [5], where only a 2D environment was studied, and in [7], which was built around user-defined gestures. We adopted the methodologies proposed in Bangor et al [2] and Schrepp et al [3], and performance scores, aiming at answering the research question: "What is the impact of game type (2D versus 3D) on system usability, user experience, and learning outcomes in ASL learning?".…”