3D Skeletal Joints-Based Hand Gesture Spotting and Classification

Nguyen, Ngoc-Hoang; Phan, Tran-Dac-Thinh; Kim, Soo-Hyung; Yang, Hyung-Jeong; Lee, Guee-Sang

doi:10.3390/app11104689

Cited by 7 publications

(4 citation statements)

References 35 publications

(66 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, BW algorithm modifies the new HMM parameters to have the highest likelihood under the condition PO jλ ðÞ . The α 0 and β 0 can be calculated using the recursive equations of ( 8) and (12) given the initial parameters λ ¼ A, B, π ðÞ . Eqs.…”

Section: Estimation Problemmentioning

confidence: 99%

“…Because segmentation and recognition are tuned concurrently during recognition using HMMs, the use of HMMs increases the effectiveness of recognition-based segmentation. The two categories of gesture are communicative gestures (also known as key gestures or meaningful gestures) and noncommunicative gestures (i.e., garbage gesture or transition gesture) [9][10][11][12]. In other words, as indicated in Figure 1, a natural gesture comprises three phases: pre-, key-, and postgesture.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hidden Markov Models for Pattern Recognition

Alwateer¹,

Elmezain²,

Farsi³

et al. 2023

Markov Model - Theory and Applications

View full text Add to dashboard Cite

The most widely used recognition algorithm for gesture recognition is Hidden Markov Models (HMMs). Hidden Markov Models are mathematical models of the stochastic process which generates a sequence of observations according to the previously stored information. Statistical approach has many advantages in HMMs like rich mathematical framework, powerful learning and decoding methods, good sequences handling capabilities, flexible topology for the statistical phonology and the syntax. The disadvantages lie in the poor discrimination between the models and in unrealistic assumptions that must be made to construct the HMMs theory, namely the independence of the successive feature frames (i.e. input vectors) and the first order Markov process. The developed algorithms in the statistical framework which uses HMMs are rich and powerful in real-time situations. In addition, Hidden Markov Models are the widely used in practice to implement gesture recognition and understanding systems. In the Markov chain, every state of the model can only observe a single symbol. However, all states in Hidden Markov Models topology can observe one symbol out of a distinct gesture. The probability of observing a symbol for each state is stored in the observation probability distribution matrix. For example, the observation probability of a symbol in the state s1 is considered as the probability to emit the symbol. So, in other words the observation probability distribution is named emission distribution in the recognition task. Furthermore, HMMs states are called hidden for the following reasons. Firstly, the decision of observing a symbol represents the second process. Secondly, the emitter of an HMM only emits the observed symbol. Finally, the emitting states are unknown since the current states are based on the previous states. In the gesture recognition, HMMs are very well known and more flexible due to their stochastic nature.

show abstract

Section: Estimation Problemmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Hidden Markov Models for Pattern Recognition

Alwateer¹,

Elmezain²,

Farsi³

et al. 2023

Markov Model - Theory and Applications

View full text Add to dashboard Cite

show abstract

“…Finally, the extracted features are transferred to a Bidirectional Gated Recurrent Unit (BiGRU) network to classify gestures. Nguyen et al [24] proposed a new continuous dynamic gesture recognition method. They use a gesture localization module to segment a video sequence of continuous gestures into individual gestures.…”

Section: Unbound Gesture Recognitionmentioning

confidence: 99%

Wi-GC: A Deep Spatiotemporal Gesture Recognition Method Based on Wi-Fi Signal

et al. 2022

View full text Add to dashboard Cite

Wireless sensing has been increasingly used in smart homes, human–computer interaction and other fields due to its comprehensive coverage, non-contact and absence of privacy leakage. However, most existing methods are based on the amplitude or phase of the Wi-Fi signal to recognize gestures, which provides insufficient recognition accuracy. To solve this problem, we have designed a deep spatiotemporal gesture recognition method based on Wi-Fi signals, namely Wi-GC. The gesture-sensitive antennas are selected first and the fixed antennas are denoised and smoothed using a combined filter. The consecutive gestures are then segmented using a time series difference algorithm. The segmented gesture data is fed into our proposed RAGRU model, where BAGRU extracts temporal features of Channel State Information (CSI) sequences and RNet18 extracts spatial features of CSI amplitudes. In addition, to pick out essential gesture features, we introduce an attention mechanism. Finally, the extracted spatial and temporal characteristics are fused and input into softmax for classification. We have extensively and thoroughly verified the Wi-GC method in a natural environment and the average gesture recognition rate of the Wi-GC way is between 92–95.6%, which has strong robustness.

show abstract

“…Existing hand action recognition methods can be divided into two kinds of mainstream: image sequence-based methods [2][3][4][5] and hand skeleton sequence-based methods [6][7][8] according to input type. RGB or RGB-D image sequence is used as input in the image sequence-based method.…”

Section: Introductionmentioning

confidence: 99%

Global Correlation Enhanced Hand Action Recognition Based on NST-GCN

Yang

et al. 2022

Electronics

View full text Add to dashboard Cite

Hand action recognition is an important part of intelligent monitoring, human–computer interaction, robotics and other fields. Compared with other methods, the hand action recognition method using skeleton information can ignore the error effects caused by complex background and movement speed changes, and the computational cost is relatively small. The spatial-temporal graph convolution networks (ST-GCN) model has excellent performance in the field of skeleton-based action recognition. In order to solve the problem of the root joint and the further joint not being closely connected, resulting in a poor hand-action-recognition effect, this paper firstly uses the dilated convolution to replace the standard convolution in the temporal dimension. This is in order to process the time series features of the hand action video, which increases the receptive field in the temporal dimension and enhances the connection between features. Then, by adding non-physical connections, the connection between the joints of the fingertip and the root of the finger is established, and a new partition strategy is adopted to strengthen the hand correlation of each joint point information. This helps to improve the network’s ability to extract the spatial-temporal features of the hand. The improved model is tested on public datasets and real scenarios. The experimental results show that compared with the original model, the 14-category top-1 and 28-category top-1 evaluation indicators of the dataset have been improved by 4.82% and 6.96%. In the real scene, the recognition effect of the categories with large changes in hand movements is better, and the recognition results of the categories with similar trends of hand movements are poor, so there is still room for improvement.

show abstract

3D Skeletal Joints-Based Hand Gesture Spotting and Classification

Cited by 7 publications

References 35 publications

Hidden Markov Models for Pattern Recognition

Hidden Markov Models for Pattern Recognition

Wi-GC: A Deep Spatiotemporal Gesture Recognition Method Based on Wi-Fi Signal

Global Correlation Enhanced Hand Action Recognition Based on NST-GCN

Contact Info

Product

Resources

About