A novel approach for dynamic hand gesture recognition using contour-based similarity images

Nasri, Saeed; Behrad, Alireza; Razzazi, Farbod

doi:10.1080/00207160.2014.915958

Cited by 17 publications

(13 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The method is evaluated on the constructed small-scale sign language data set and met the real-time requirements of the small-scale sign language recognition system. Nasri et al proposed two novel representations for the recognition of moving hand gestures, one is the contour-based similarity images (CBSIs) [12] and the other is spatio-temporal 3D surfaces [13]. Both of them can simultaneously divide the continuous gestures into disjointed gestures and recognize them.…”

Section: Related Workmentioning

confidence: 99%

Multimodal Spatiotemporal Networks for Sign Language Recognition

Zhang

Meng

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Different from other human behaviors, sign language has the characteristics of limited local motion of upper limb and meticulous hand action. Some sign language gestures are ambiguous in RGB video due to the influence of lighting and background color, which affects the recognition accuracy. We propose a multimodal deep learning architecture for sign language recognition which effectively combines RGB-D input and two-stream spatiotemporal networks. Depth videos, as an effective compensation of RGB input, can supply additional distance information about the signer's hands. A novel sampling method called ARSS (Aligned Random Sampling in Segments) is put forward to select and align optimal RGB-D video frames, which improves the capacity utilization of multimodal data and reduces the redundancy. We get the hand ROI by joints information of RGB data for local focus in spatial stream. D-shift Net is proposed as depth motion feature extraction in temporal stream, which fully utilizes three dimensional motion information of the sign language. Both streams are fused by convolutional fusion layer to get complementary features. Our approach explored the multimodal information and enhanced the recognition precision. It obtains the state-the-of-art performance on the datasets of CSL (96.7%) and IsoGD (63.78%). INDEX TERMS Sign language recognition, two-stream network, motion features, multimodal data.

show abstract

Section: Related Workmentioning

confidence: 99%

Multimodal Spatiotemporal Networks for Sign Language Recognition

Zhang

Meng

et al. 2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…However, in the real-time applications of the hand gesture recognition system, the hand gestures are usually continuous and dynamic. To this end, some researchers focus on hand gesture recognition with continuous and dynamic hand gestures [20][21][22][23]. In [20], the authors obtain the start and end times of each single gesture by a near-zero speed detection method.…”

Section: Introductionmentioning

confidence: 99%

A Novel Detection and Recognition Method for Continuous Hand Gesture Using FMCW Radar

et al. 2020

View full text Add to dashboard Cite

In this paper, a novel method for continuous hand gesture detection and recognition is proposed based on a frequency modulated continuous wave (FMCW) radar. Firstly, we adopt the 2-Dimensional Fast Fourier Transform (2D-FFT) to estimate the range and Doppler parameters of the hand gesture raw data, and construct the range-time map (RTM) and Doppler-time map (DTM). Meanwhile, we apply the Multiple Signal Classification (MUSIC) algorithm to calculate the angle and construct the angletime map (ATM). Secondly, a hand gesture detection method is proposed to segment the continuous hand gestures using a decision threshold. Thirdly, the central time-frequency trajectory of each hand gesture spectrogram is clustered using the k-means algorithm, and then the Fusion Dynamic Time Warping (FDTW) algorithm is presented to recognize the hand gestures. Finally, experiments show that the accuracy of the proposed hand gesture detection method can reach 96.17%. The hand gesture average recognition accuracy of the proposed FDTW algorithm is 95.83%, while its time complexity is reduced by more than 50%. INDEX TERMS FMCW radar, continuous hand gesture recognition, detection, time-frequency trajectory, FDTW.

show abstract

“…However, the color-based methods may fail when the background contains other objects with the color of skin. Hence some color-independent features are employed, such as edges [5], contours [6] and textures [7]. In order to improve the recognition accuracy under complex backgrounds, these features are usually combined to locate hands.…”

Section: Introductionmentioning

confidence: 99%

A Real-Time Hand Detection System during Hand over Face Occlusion

Xu¹,

Zhang²

2015

IJMUE

View full text Add to dashboard Cite

This paper devises a vision-based hand detection system which can handle the situation of the hand and face occlusion. Firstly, human face is coarsely located by skin color detection and ellipse template matching and further determined by using the grayscales and positions of human eyes. Secondly, when the hand occludes the partial region of the face, a novel hand detection method based on Camshift tracking algorithm, force field method and Sobel edge extraction is developed to segment the hand. Finally, the positions of segmented hands are sent to the computer for controlling the cursor movement. In order to reduce the cursor jitter caused by wrong hand detection and generate a smooth trajectory of the cursor, the coordinates of hand positions are corrected by combining the least squares fitting method of orthogonal polynomial and an adaptive Catmull-Rom interpolation algorithm. Experimental results showed that the proposed method could detect hands accurately with the run time of 0.08s per frame and demonstrated a significant improvement in performance when the hand enters the face region. Moreover, our system accomplished smooth movements of the cursor by the vision-based hand detection.

show abstract

A novel approach for dynamic hand gesture recognition using contour-based similarity images

Cited by 17 publications

References 14 publications

Multimodal Spatiotemporal Networks for Sign Language Recognition

Multimodal Spatiotemporal Networks for Sign Language Recognition

A Novel Detection and Recognition Method for Continuous Hand Gesture Using FMCW Radar

A Real-Time Hand Detection System during Hand over Face Occlusion

Contact Info

Product

Resources

About