Spatio-temporal 3D surface matching for hand gesture recognition using ICP algorithm

Nasri, Saeed; Behrad, Alireza; Razzazi, Farbod

doi:10.1007/s11760-013-0558-7

Cited by 8 publications

(6 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, the proposed method will be evaluated systematically on two public datasets: Chinese Sign Language dataset (CSL) [40] and the ChaLearn LAP large-scale isolated gesture dataset (IsoGD) [14]. First, the two datasets will be described briefly.…”

Section: Experimental Results and Analysismentioning

confidence: 99%

“…In addition to traditional feature extraction methods, RGB-D detection methods combined with deep learning convolutional neural network also develop fast. Multi-scale deep learning [14] adopts multimode convolutional neural network to integrate various modal data and learn representations of multiple spatial and temporal scales. The data modes integrated by the algorithm include the grayscale and depth video, as well as the joint pose information extracted from the depth map.…”

Section: A Rgb-dmentioning

confidence: 99%

“…Unlike the global-local mode [14], [36], this paper keeps the left and right hand ROI in the same RGB image without dividing into two inputs to preserve their relative position and interaction.…”

Section: A Arssmentioning

confidence: 99%

See 2 more Smart Citations

Multimodal Spatiotemporal Networks for Sign Language Recognition

Zhang

Meng

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Different from other human behaviors, sign language has the characteristics of limited local motion of upper limb and meticulous hand action. Some sign language gestures are ambiguous in RGB video due to the influence of lighting and background color, which affects the recognition accuracy. We propose a multimodal deep learning architecture for sign language recognition which effectively combines RGB-D input and two-stream spatiotemporal networks. Depth videos, as an effective compensation of RGB input, can supply additional distance information about the signer's hands. A novel sampling method called ARSS (Aligned Random Sampling in Segments) is put forward to select and align optimal RGB-D video frames, which improves the capacity utilization of multimodal data and reduces the redundancy. We get the hand ROI by joints information of RGB data for local focus in spatial stream. D-shift Net is proposed as depth motion feature extraction in temporal stream, which fully utilizes three dimensional motion information of the sign language. Both streams are fused by convolutional fusion layer to get complementary features. Our approach explored the multimodal information and enhanced the recognition precision. It obtains the state-the-of-art performance on the datasets of CSL (96.7%) and IsoGD (63.78%). INDEX TERMS Sign language recognition, two-stream network, motion features, multimodal data.

show abstract

Section: Experimental Results and Analysismentioning

confidence: 99%

Section: A Rgb-dmentioning

confidence: 99%

See 1 more Smart Citation

Multimodal Spatiotemporal Networks for Sign Language Recognition

Zhang

Meng

et al. 2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Most of these studies extract specific features and then use machine learning algorithms to classify the SL images. Many different SL have been used in these studies, namely American SL [1][2][3][4][5][6][7][8] , Arabic SL [9][10][11][12] , British SL 13 , Chinese SL 14 , German SL 15,16 , Indian SL 17 , Irish SL 18 , Pakistani SL [19][20][21][22][23][24] , Persian SL 25 , and more in combination such as American & German SL 26 , American & Thai SL 27 and American & Indian SL 28 .…”

Section: Literature Reviewmentioning

confidence: 99%

Vision-based Pakistani Sign Language Recognition Using Bag-of-Words and Support Vector Machines

Mirza

Munaf

Ali

et al. 2022

Preprint

View full text Add to dashboard Cite

In order to perform their daily activities, a person is required to communicating with others. This can be a major obstacle for the deaf population of the world, who communicate using sign languages (SL). Pakistani Sign Language (PSL) is used by more than 250,000 deaf Pakistanis. Developing a SL recognition system would greatly facilitate these people. This study aimed to collect data of static and dynamic PSL alphabets and to develop a vision-based system for their recognition using Bag-of-Words (BoW) and Support Vector Machine (SVM) techniques. A total of 5,120 images for 36 static PSL alphabet signs and 353 videos with 45,224 frames for 3 dynamic PSL alphabet signs were collected from 10 native signers of PSL. The developed system used the collected data as input, resized the data to various scales and converted the RGB images into grayscale. The resized grayscale images were segmented using Thresholding technique and features were extracted using Speeded Up Robust Feature (SURF). The obtained SURF descriptors were clustered using K-means clustering. A BoW was obtained by computing the Euclidean distance between the SURF descriptors and the clustered data. The codebooks were divided into training and testing using 5-fold cross validation. The highest overall classification accuracy for static PSL signs was 97.80% at 750×750 image dimensions and 500 Bags. For dynamic PSL signs a 96.53% accuracy was obtained at 480×270 video resolution and 200 Bags.

show abstract

“…The goal of gesture recognition is to use manpower as a direct input device, eliminating the need for intermediate media and controlling the machine directly through defined gestures [2]. In interactive gestures of human-machine gestures, the purpose is to make the machine understand the meaning of gestures in front of it [3]. During this period are often required rivals in real-time detection, tracking, identification.…”

Section: Introductionmentioning

confidence: 99%

Jointly network: a network based on CNN and RBM for gesture recognition

Cheng

Sun

et al. 2018

Neural Comput & Applic

View full text Add to dashboard Cite

Hand belongs to non-rigid objects and is rich in variety, making gesture recognition more difficult. The essence of dynamic gesture recognition is the classification and recognition of single-frame still images. Therefore, this paper mainly focuses on static gesture recognition. At present, there are some problems in gesture recognition, such as accuracy, real-time or poor robustness. To solve the above problems, in this paper, the Kinect sensor is used to obtain the color and depth gesture samples, and the gesture samples are processed. On this basis, a jointly network of CNN and RBM is proposed for gesture recognition. It mainly uses superposed network of multiple RBMs to carry out unsupervised feature extraction and combined with supervised feature extraction of CNN. Finally, these two features are combined to classify them. The simulation results show that the proposed jointly network has a better performance in identifying simple background gesture samples, and the recognition capability of gesture samples in complex background needs to be improved.

show abstract

Spatio-temporal 3D surface matching for hand gesture recognition using ICP algorithm

Cited by 8 publications

References 25 publications

Multimodal Spatiotemporal Networks for Sign Language Recognition

Multimodal Spatiotemporal Networks for Sign Language Recognition

Vision-based Pakistani Sign Language Recognition Using Bag-of-Words and Support Vector Machines

Jointly network: a network based on CNN and RBM for gesture recognition

Contact Info

Product

Resources

About