HGR-ViT: Hand Gesture Recognition with Vision Transformer

Tan, Chun Keat; Lim, Kian Ming; Chang, Roy Kwang Yang; Lee, Chin Poo; Alqahtani, Ali

doi:10.3390/s23125555

Cited by 8 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This model recognized the input gesture model. These experiments confirmed that the proposed recognition model had good model performance in various data sets [5]. P. S. Neethu et al found that combination gestures were very complex and difficult for machines to classify.…”

Section: Related Worksupporting

confidence: 75%

Hand Function Rehabilitation Training Robot Based on Ycbcr and CNN

Shan,

Zhou

2024

IEEE Access

View full text Add to dashboard Cite

Robot technology shows broad application prospects in rehabilitation medicine, especially in hand rehabilitation. Hand function plays an important role that cannot be ignored in daily life, and its key properties are reflected in multiple levels. From basic life skills to occupational needs, healthy hand function is indispensable. Hand function is the foundation for performing daily life skills, including activities such as self-care, eating, dressing, and grooming. The ability to freely use hand functions is directly related to an individual's quality of life and independence. This study proposed a gesture recognition algorithm by fusing Ycbcr color space and convolutional neural network. The method first converted gesture images and recognizes them through the converted images. Then, a hand function rehabilitation training robot based on Ycbcr and CNN was designed, which provided rehabilitation treatment for patients with impaired hand function. These experiments confirmed that when the data set size was 500, the signal-to-noise ratios of YOLOV3, YOLOV3-SPP, YOLOV4, and hybrid algorithms were 27.5dB, 32.7dB, 34.8dB, and 41.2dB, respectively. Their inter-section of union values were 0.53, 0.64, 0.77, and 0.89, respectively. These results confirm that the proposed hybrid algorithm model has good model performance in various algorithms.

show abstract

Section: Related Worksupporting

confidence: 75%

Hand Function Rehabilitation Training Robot Based on Ycbcr and CNN

Shan,

Zhou

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Tan et al (2023) [ 14 ] introduced HGR-ViT, a Vision Transformer (ViT) model incorporating an attention mechanism designed for hand gesture recognition. The approach began by dividing a given hand gesture image into fixed-size patches.…”

Section: Related Workmentioning

confidence: 99%

LAVRF: Sign language recognition via Lightweight Attentive VGG16 with Random Forest

Ewe,

Lee,

Lim

et al. 2024

PLoS ONE

Self Cite

View full text Add to dashboard Cite

Sign language recognition presents significant challenges due to the intricate nature of hand gestures and the necessity to capture fine-grained details. In response to these challenges, a novel approach is proposed—Lightweight Attentive VGG16 with Random Forest (LAVRF) model. LAVRF introduces a refined adaptation of the VGG16 model integrated with attention modules, complemented by a Random Forest classifier. By streamlining the VGG16 architecture, the Lightweight Attentive VGG16 effectively manages complexity while incorporating attention mechanisms that dynamically concentrate on pertinent regions within input images, resulting in enhanced representation learning. Leveraging the Random Forest classifier provides notable benefits, including proficient handling of high-dimensional feature representations, reduction of variance and overfitting concerns, and resilience against noisy and incomplete data. Additionally, the model performance is further optimized through hyperparameter optimization, utilizing the Optuna in conjunction with hill climbing, which efficiently explores the hyperparameter space to discover optimal configurations. The proposed LAVRF model demonstrates outstanding accuracy on three datasets, achieving remarkable results of 99.98%, 99.90%, and 100% on the American Sign Language, American Sign Language with Digits, and NUS Hand Posture datasets, respectively.

show abstract

“…They are also the most intuitive in a smart environment, as they can interact with devices as well as be universally used in various applications. User demands about hand gesture recognition have recently been continuously increased, and the research is being actively developed with regard to improving the gesture-recognition rate [ 7 , 8 ]. The hand gesture system is being applied to various devices, such as drones, robots, and smart home devices, as hand gesture recognition develops [ 9 , 10 ].…”

Section: Introductionmentioning

confidence: 99%

Hand Gesture Recognition Using FSK Radar Sensors

Yang,

Kim,

Jung

et al. 2024

Sensors

View full text Add to dashboard Cite

Hand gesture recognition, which is one of the fields of human–computer interaction (HCI) research, extracts the user’s pattern using sensors. Radio detection and ranging (RADAR) sensors are robust under severe environments and convenient to use for hand gestures. The existing studies mostly adopted continuous-wave (CW) radar, which only shows a good performance at a fixed distance, which is due to its limitation of not seeing the distance. This paper proposes a hand gesture recognition system that utilizes frequency-shift keying (FSK) radar, allowing for a recognition method that can work at the various distances between a radar sensor and a user. The proposed system adopts a convolutional neural network (CNN) model for the recognition. From the experimental results, the proposed recognition system covers the range from 30 cm to 180 cm and shows an accuracy of 93.67% over the entire range.

show abstract

HGR-ViT: Hand Gesture Recognition with Vision Transformer

Cited by 8 publications

References 34 publications

Hand Function Rehabilitation Training Robot Based on Ycbcr and CNN

Hand Function Rehabilitation Training Robot Based on Ycbcr and CNN

LAVRF: Sign language recognition via Lightweight Attentive VGG16 with Random Forest

Hand Gesture Recognition Using FSK Radar Sensors

Contact Info

Product

Resources

About