Two-Stream Convolutional Network for Dynamic Hand Gesture Recognition Using Convolutional Long Short-Term Memory Networks

Nguyen, Phuong Chi; Luong, Tien Ngoc

doi:10.15625/2525-2518/58/4/14742

Cited by 7 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nguyen et al proposed that the dance movement can be regarded as a continuous set of postures, so it needs to be represented by the information of multiple images. In order to extract human motion features from the preprocessed images, not only should the feature information in the image at a certain time be extracted, but also the correlation between the motion features of adjacent images should be established [ 15 ]. Thomas et al state that the purpose of human dance-specific action recognition is to analyze and understand human body actions and behaviors in video.…”

Section: Related Workmentioning

confidence: 99%

[Retracted] Dance‐Specific Action Recognition Method Based on Double‐Stream CNN in Complex Environment

Jin

2022

Journal of Environmental and Public Health

View full text Add to dashboard Cite

Technology for dance-specific motion recognition is widely used in many industries, but Chinese research in this area is still in its early stages. Recognizing specific dance movements is the key to learning about and comprehending human actions and behaviors. The fault-tolerant feature of standardized sign language recognition is extended under the condition of small sample sizes, but the recognition accuracy remains a challenge. This issue needs to be resolved by fusing the essential details of particular dance movements. A dual-stream convolution neural network is suggested in this paper to investigate the recognition of particular dance movements. In this paper, a dual-stream convolution neural network is used to study the recognition of particular dance movements. The time spent by this algorithm gradually increases as the number of people in the image does, but only slightly. The algorithms proposed by Bergonzoni (2017) and Liu et al. (2021) both experience linear increases in running time as the population grows. In contrast, the running time of the algorithm in this study essentially increases negligibly. It has become a problem deserving in-depth study. Double-stream convolution neural network improves the practical value and technical complexity of dance motion automatic generation technology in art and cultural heritage protection, dance teaching, dance video retrieval, and dance arrangement.

show abstract

Section: Related Workmentioning

confidence: 99%

[Retracted] Dance‐Specific Action Recognition Method Based on Double‐Stream CNN in Complex Environment

Jin

2022

Journal of Environmental and Public Health

View full text Add to dashboard Cite

show abstract

“…To summarize, it is evident that datasets depicting more realistic conditions have a lower accuracy: datasets IPN Hand and IsoGD received the lowest accuracy with 62.14% [42], 67.71% [43], 82.90% and 85.10% [34]. All studies using less complex datasets achieved an accuracy above 90.00% [36], [38], [42].…”

Section: B Gesture Recognition With Deep Learningmentioning

confidence: 90%

“…Nguyen et al [36] further developed this approach on two streams of CNN architecture, allowing the system to consider both the spatial and temporal aspects of the hand gestures in videos. Hand gestures often involve complex movements and disproportions of the hand.…”

Section: B Gesture Recognition With Deep Learningmentioning

confidence: 99%

See 1 more Smart Citation

A Novel Hybrid Deep Learning Architecture for Dynamic Hand Gesture Recognition

Hax,

Penava,

Krodel

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Hand gestures are a form of natural communication used in human-computer interaction, however, when gestures are video-based, extraction of features for classification is complex. Current machine learning models struggle to achieve high accuracies when using videos recorded in realistic environments. In this work, we propose a hybrid architecture consisting of a recurrent neural network (RNN), including a long short-term memory layer, on top of a convolutional neural network, to recognize dynamic hand gestures recorded in realistic environments. We used a dataset of 6 dynamic hand gestures: scroll-left, scroll-right, scroll-up, scroll-down, zoom-in, and zoom-out. Our implemented inception-v3 model extracted features and provided the wrapped frame-feature map as input for the RNN, which performs the final classification. The proposed model classifies gestures with an average accuracy of 83.66%. By doing so, we intend to narrow the disparity between realistic environments and high accuracy. Finally, we compare the accuracy of our proposed dynamic hand gesture recognition model with that of the benchmark.

show abstract

“…Nguyen et al [19] proposed a two-stream convolution network model on 6 classes out of 25 using the 20BN-jester dataset. MobileNet-V2 followed by LSTM was used for spatio-temporal features extraction.…”

Section: Related Workmentioning

confidence: 99%

Dynamic Hand Gesture Recognition Using 3D-CNN and LSTM Networks

Rehman¹,

Ahmed²,

Khan³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Recognition of dynamic hand gestures in real-time is a difficult task because the system can never know when or from where the gesture starts and ends in a video stream. Many researchers have been working on visionbased gesture recognition due to its various applications. This paper proposes a deep learning architecture based on the combination of a 3D Convolutional Neural Network (3D-CNN) and a Long Short-Term Memory (LSTM) network. The proposed architecture extracts spatial-temporal information from video sequences input while avoiding extensive computation. The 3D-CNN is used for the extraction of spectral and spatial features which are then given to the LSTM network through which classification is carried out. The proposed model is a light-weight architecture with only 3.7 million training parameters. The model has been evaluated on 15 classes from the 20BN-jester dataset available publicly. The model was trained on 2000 video-clips per class which were separated into 80% training and 20% validation sets. An accuracy of 99% and 97% was achieved on training and testing data, respectively. We further show that the combination of 3D-CNN with LSTM gives superior results as compared to MobileNetv2 + LSTM.

show abstract

Two-Stream Convolutional Network for Dynamic Hand Gesture Recognition Using Convolutional Long Short-Term Memory Networks

Cited by 7 publications

References 9 publications

[Retracted] Dance‐Specific Action Recognition Method Based on Double‐Stream CNN in Complex Environment

[Retracted] Dance‐Specific Action Recognition Method Based on Double‐Stream CNN in Complex Environment

A Novel Hybrid Deep Learning Architecture for Dynamic Hand Gesture Recognition

Dynamic Hand Gesture Recognition Using 3D-CNN and LSTM Networks

Contact Info

Product

Resources

About