Block-based histogram of optical flow for isolated sign language recognition

Lim, Kian Ming; Tan, Alan Wee Chiat; Tan, Shing Chiang

doi:10.1016/j.jvcir.2016.07.020

Cited by 42 publications

(20 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, other body parts, such as face and arms possess similar skin color information; thus they can be erroneously recognized and extracted along with hands. As a result, recent hand detection methods rely also on face detection and background subtraction in order to identify only the moving parts of a scene [7] [8]. Moreover, hand detection methods usually employ tracking techniques, such as Kalman and particle filters in order to handle occlusion problems and achieve accurate and robust hand detection and extraction [8] [9].…”

Section: Related Workmentioning

confidence: 99%

“…As far as hand gesture classification is concerned, several methods employ the extracted hand regions and compute distances between histograms of optical flow [7] or feature covariance matrices from pixel intensities [8]. The success of Hidden Markov Models (HMM) on several tasks, such as speech and handwriting recognition, has led to their use on SLR as well.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Deep Learning Approach for Analyzing Video and Skeletal Features in Sign Language Recognition

Konstantinidis

Dimitropoulos

Daras

2018

2018 IEEE International Conference on Imaging Systems and Techniques (IST)

View full text Add to dashboard Cite

Sign language recognition (SLR) refers to the classification of signs with a specific meaning performed by the deaf and/or hearing-impaired people in their everyday communication. In this work, we propose a deep learning based framework, in which we examine and analyze the contribution of video (image and optical flow) and skeletal (body, hand and face) features in the challenging task of isolated SLR, in which each signed video corresponds to a single word. Moreover, we employ various fusion schemes in order to identify the optimal way to combine the information obtained from the various feature representations and propose a robust SLR methodology. Our experimentation on two sign language datasets and the comparison with state-of-the-art SLR methods reveals the superiority of optimally combining skeletal and video features for SLR tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

A Deep Learning Approach for Analyzing Video and Skeletal Features in Sign Language Recognition

Konstantinidis

Dimitropoulos

Daras

2018

2018 IEEE International Conference on Imaging Systems and Techniques (IST)

View full text Add to dashboard Cite

show abstract

“…Feature extraction method is used to create feature vector which contain set of features with reduced representation. Histogram of gradient method [4] [5] method is used to divide image into different blocks with cell & from cell feature extraction occurred. Magnitude of robust vector decide ensuing vector for a specific cell inside block.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning Feature Extraction using Pre-Trained Alex Net Model for Indian Sign Language Recognition

Dangarwala¹,

Hiran²

2019

IJRTE

View full text Add to dashboard Cite

Indian sign language is communicating language among deaf and dumb people of India. Hand gestures are broadly used as communication gestures among various forms of gesture. The real time classification of different signs is a challenging task due to the variation in shape and position of hands as well as due to the variation in the background which varies from person to person. There seems to be no availability of datasets resembling to Indian signs which poses a problem to the researcher. To address this problem, we design our own dataset which is formed by incorporating 1000 signs for the sign digits from 1 to 10 from 100 different people with varying backgrounds conditions by changing colour, and light illumination situations. The dataset comprises of the signs from left handed as well as right handed people. Feature extraction methodologies are studied and applied to recognition of Sign language. This paper focuses on deep learning CNN (convolution neural network) approach with pretrained model Alexnet for calculation of feature vector. Multiple SVM (Support Vector Machine) is applied to classify Indian sign language in real time surroundings. This paper also shows the comparative analysis between Deep learning feature extraction method with histogram of gradient, bag of feature and Speed up robust feature extraction method. The experimental results shown that Deep learning feature extraction using pretrained Alexnet model give accuracy of around 85% and above for the recognition of signed digit with the use of 60% training set and 40% testing set.

show abstract

“…Hand detection has been achieved by semantic segmentation and skin color detection as skin color is usually easy to distinguish [3] [4]. However, due to the fact that other body parts (e.g., face and arms) can be mistakenly recognized as hands, recent hand detection methods rely also on face detection and subtraction and background subtraction to identify only the moving parts of a scene [5] [6]. To achieve accurate and robust hand tracking, especially in cases of occlusions, previous methods employ filtering techniques, such as Kalman and particle filters [6] [7].…”

Section: Introductionmentioning

confidence: 99%

“…Most SLR systems employ the original or modified versions of HMMs on the extracted hand motions and shapes in order to accurately detect and classify hand gestures [8][9] [10]. Other successful SLR methodologies rely on distances between histograms of optical flow [5] and feature covariance matrices computed from the intensity of pixels [6], extracted from the detected hand regions.…”

Section: Introductionmentioning

confidence: 99%

Sign Language Recognition Based on Hand and Body Skeletal Data

Konstantinidis

Dimitropoulos

Daras

2018

2018 - 3dtv-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3dtv-Con)

View full text Add to dashboard Cite

Sign language recognition (SLR) is a challenging, but highly important research field for several computer vision systems that attempt to facilitate the communication among the deaf and hearing impaired people. In this work, we propose an accurate and robust deep learning-based methodology for sign language recognition from video sequences. Our novel method relies on hand and body skeletal features extracted from RGB videos and, therefore, it acquires highly discriminative for gesture recognition skeletal data without the need for any additional equipment, such as data gloves, that may restrict signer's movements. Experimentation on a large publicly available sign language dataset reveals the superiority of our methodology with respect to other state of the art approaches relying solely on RGB features.

show abstract

Block-based histogram of optical flow for isolated sign language recognition

Cited by 42 publications

References 20 publications

A Deep Learning Approach for Analyzing Video and Skeletal Features in Sign Language Recognition

A Deep Learning Approach for Analyzing Video and Skeletal Features in Sign Language Recognition

Deep Learning Feature Extraction using Pre-Trained Alex Net Model for Indian Sign Language Recognition

Sign Language Recognition Based on Hand and Body Skeletal Data

Contact Info

Product

Resources

About