Sign Language Recognition Using Convolutional Neural Networks

Pigou, Lionel; Dieleman, Sander; Kindermans, Pieter-Jan; Schrauwen, Benjamin

doi:10.1007/978-3-319-16178-5_40

Cited by 315 publications

(164 citation statements)

References 10 publications

Supporting

Mentioning

158

Contrasting

Unclassified

Order By: Relevance

“…Similar to human action recognition, sign language recognition was also influenced by the recent developments of deep learning. Pigou et al explore the idea of merging and forming CNN and RNN based architectures for gesture recognition [39], and proposed a CNN based architecture for sign language recognition [38]. Koller et al [26] proposed a frame-based CNN-HMM architecture for sign language classification problem which was trained on nearly 1 million hand images.…”

Section: Related Workmentioning

confidence: 99%

Temporal Accumulative Features for Sign Language Recognition

Kındıroğlu

Özdemir

Akarun

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

In this paper, we propose a set of features called temporal accumulative features (TAF) for representing and recognizing isolated sign language gestures. By incorporating sign language specific constructs to better represent the unique linguistic characteristic of sign language videos, we have devised an efficient and fast SLR method for recognizing isolated sign language gestures. The proposed method is an HSV based accumulative video representation where keyframes based on the linguistic movement-hold model are represented by different colors. We also incorporate hand shape information and using a small scale convolutional neural network, demonstrate that sequential modeling of accumulative features for linguistic subunits improves upon baseline classification results.

show abstract

Section: Related Workmentioning

confidence: 99%

Temporal Accumulative Features for Sign Language Recognition

Kındıroğlu

Özdemir

Akarun

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

show abstract

“…Sign language involves the use of gestures. Thus, this section introduces gesture-recognition methods, which can be divided into two categories, namely hand-gesture recognition [2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19]21] and non-hand-gesture recognition [22,23]. Because the proposed method is a hand-gesture recognition method, this section introduces several competitive approaches from among the developed hand-gesture recognition methods.…”

Section: Related Workmentioning

confidence: 99%

“…For effective sign-language recognition, previous studies have addressed the following three difficulties [2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19]. First, the required data size is large because the data for one sign action usually consists of dozens of images.…”

Section: Introductionmentioning

confidence: 99%

Human-like sign-language learning method using deep learning

Kim

et al. 2018

ETRI Journal

View full text Add to dashboard Cite

This paper proposes a human‐like sign‐language learning method that uses a deep‐learning technique. Inspired by the fact that humans can learn sign language from just a set of pictures in a book, in the proposed method, the input data are pre‐processed into an image. In addition, the network is partially pre‐trained to imitate the preliminarily obtained knowledge of humans. The learning process is implemented with a well‐known network, that is, a convolutional neural network. Twelve sign actions are learned in 10 situations, and can be recognized with an accuracy of 99% in scenarios with low‐cost equipment and limited data. The results show that the system is highly practical, as well as accurate and robust.

show abstract

“…are beneficial for the gesture recognition task showing experimentally how they outperform low-level appearance features (Yao et al, 2011). More complex artificial neural network models such as convolutional neural networks (CNNs) have been also applied in multi-modal gesture recognition (Pigou, Dieleman, Kindermans, & Schrauwen, 2014). In this case, the feature extraction process is implicit in the classification model and a segmentation process using skeletal information was performed.…”

Section: Related Workmentioning

confidence: 99%

A long short‐term memory based Schaeffer gesture recognition system

et al. 2017

View full text Add to dashboard Cite

In this work, a Schaeffer language recognition system is proposed in order to help autistic children overcome communicative disorders. Using Schaeffer language as a speech and language therapy, improves children communication skills and at the same time the understanding of language productions. Nevertheless, the teaching process of children in performing gestures properly is not straightforward. For this purpose, this system will teach children with autism disorder the correct way to communicate using gestures in combination with speech reproduction. The main purpose is to accelerate the learning process and increase children interest by using a technological approach. Several recurrent neural network‐based approaches have been tested, such as vanilla recurrent neural networks, long short‐term memory networks,and gated recurrent unit‐based models. In order to select the most suitable model, an extensive comparison has been conducted reporting a 93.13% classification success rate over a subset of 25 Schaeffer gestures by using an long short‐term memory‐based approach. Our dataset consists on pose‐based features such as angles and euclidean distances extracted from the raw skeletal data provided by a Kinect v2 sensor.

show abstract

Sign Language Recognition Using Convolutional Neural Networks

Cited by 315 publications

References 10 publications

Temporal Accumulative Features for Sign Language Recognition

Temporal Accumulative Features for Sign Language Recognition

Human-like sign-language learning method using deep learning

A long short‐term memory based Schaeffer gesture recognition system

Contact Info

Product

Resources

About