Fast and robust dynamic hand gesture recognition via key frames extraction and feature fusion

Tang, Hao; Liu, Hong; Wei, Xiao; Sebe, Nicu

doi:10.1016/j.neucom.2018.11.038

Cited by 113 publications

(42 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In feature based approach, a set of key frames have been extracted from the video sequences through a fast and efficient method based on image entropy and density clustering as proposed in [7] . The keyframe extraction eliminates the redundant information and makes all the videos with equal numbers of frames.…”

Section: Discussionmentioning

confidence: 99%

Hand gestures for emergency situations: A video dataset based on words from Indian sign language

Adithya

Rajesh

2020

Data in Brief

View full text Add to dashboard Cite

Automatic sign language recognition provides better services to the deaf as it avoids the existing communication gap between them and the rest of the society. Hand gestures, the primary mode of sign language communication, plays a key role in improving sign language recognition. This article presents a video dataset of the hand gestures of Indian sign language (ISL) words used in emergency situations. The videos of eight ISL words have been collected from 26 individuals (including 12 males and 14 females) in the age group of 22 to 26 years with two samples from each individual in an indoor environment with normal lighting conditions. Such a video dataset is highly needed for automatic recognition of emergency situations from the sign language for the benefit of the deaf. The dataset is useful for the researchers working on vision based sign language recognition (SLR) as well as hand gesture recognition (HGR). Moreover, support vector machine based classification and deep learning based classification of the emergency gestures has been carried out and the base classification performance shows that the database can be used as a benchmarking dataset for developing novel and improved techniques for recognizing the hand gestures of emergency words in Indian sign language.

show abstract

Section: Discussionmentioning

confidence: 99%

Hand gestures for emergency situations: A video dataset based on words from Indian sign language

Adithya

Rajesh

2020

Data in Brief

View full text Add to dashboard Cite

show abstract

“…For the comparison, four different baselines were selected: (1) entropy-based (with color histograms) [ 31 ], (2) color-based [ 17 ], (3) sliding window based (with gist and sift features) [ 18 ], and (4) our method without deduplication optimization.…”

Section: Methodsmentioning

confidence: 99%

“…Table 2 shows the results. The entropy-based [ 31 ] method calculated the entropy value of the frame to perform key frames selection, and the color-based [ 17 ] method extract color characteristics to obtain temporal segment on the video. The sliding window based [ 18 ] extracts features of frames by using gist and sift features and then use the sliding window to extract key frames.…”

Section: Methodsmentioning

confidence: 99%

Video Summarization Based on Mutual Information and Entropy Sliding Window Method

Zhang

et al. 2020

Entropy

View full text Add to dashboard Cite

This paper proposes a video summarization algorithm called the Mutual Information and Entropy based adaptive Sliding Window (MIESW) method, which is specifically for the static summary of gesture videos. Considering that gesture videos usually have uncertain transition postures and unclear movement boundaries or inexplicable frames, we propose a three-step method where the first step involves browsing a video, the second step applies the MIESW method to select candidate key frames, and the third step removes most redundant key frames. In detail, the first step is to convert the video into a sequence of frames and adjust the size of the frames. In the second step, a key frame extraction algorithm named MIESW is executed. The inter-frame mutual information value is used as a metric to adaptively adjust the size of the sliding window to group similar content of the video. Then, based on the entropy value of the frame and the average mutual information value of the frame group, the threshold method is applied to optimize the grouping, and the key frames are extracted. In the third step, speeded up robust features (SURF) analysis is performed to eliminate redundant frames in these candidate key frames. The calculation of Precision, Recall, and Fmeasure are optimized from the perspective of practicality and feasibility. Experiments demonstrate that key frames extracted using our method provide high-quality video summaries and basically cover the main content of the gesture video.

show abstract

“…The algorithm has to be based on clear gesture contours, it will get poor accuracy when the gesture is blurred because of motion. Tang et al used a keyframe extraction approach to recognise the dynamic gesture [23]. Firstly, they extracted keyframes from a video based on image entropy and density clustering.…”

Section: Related Workmentioning

confidence: 99%

Real‐time multi‐trajectory matching for dynamic hand gesture recognition

Jian

2020

IET Image Processing

View full text Add to dashboard Cite

Fast and robust dynamic hand gesture recognition via key frames extraction and feature fusion

Cited by 113 publications

References 60 publications

Hand gestures for emergency situations: A video dataset based on words from Indian sign language

Hand gestures for emergency situations: A video dataset based on words from Indian sign language

Video Summarization Based on Mutual Information and Entropy Sliding Window Method

Real‐time multi‐trajectory matching for dynamic hand gesture recognition

Contact Info

Product

Resources

About