A CNN sign language recognition system with single & double-handed gestures

Abstract

This work aims at presenting a novel Computer Vision approach in the development of a real-time, web-camera based, British Sign Language recognition system. A literature review focused on current (1) state of sign language recognition systems and (2) techniques used is conducted. This review process is used as a foundation on which a Convolutional Neural Network (CNN) based system is designed and then implemented. A bespoke British Sign Language datasetcontaining 11,875 images -is then performed to train and test the CNN which is used for the classification of human hand performed gestures. Finally, the CNN architecture recognized 19 static British Sign Language gestures, incorporating both single and double-handed gestures. During testing, the system achieved an average recognition accuracy of 89%.

Raspberry-PI based Design of An Interactive Smart Mirror for Daily Life

“…Such a system could also be integrated with a set of sensors and other smart home devices or connected to an Ambient Assisted Living or medical system [16,17]. In this context, it is reasonable to also foresee the integration with gesture recognition systems making the mirror more intuitive when interacting with the end-user [18,19]. Hardware could be changed to increase size and reduce depth; increasing usability and functionality while reducing noticeability.…”

Indian Sign Language Recognition Using CNN Inception V3 - A Review

“…Therefore, by combining different fundamental elements, complicated meanings can be explained. [3]. The automatic recognition of gestures performed by the human is a complex problem, and it is not yet been solved completely.…”

Development of a Wearable Sensor Glove for Real-Time Sign Language Translation

“…In these systems, frame images from the recorded videos are used as an input for the sign language translation system, which requires a computer to translate the hand gestures by using image processing algorithms. Vision-based systems can be implemented easily using a web camera [1][2][3][4], multiple cameras [5], and a smartphone camera [6][7]. Several researchers had utilised active techniques in vision-based systems using off-the-shelf tools such as Microsoft's Kinect [8][9][10], Intel's RealSense [11], and Leap Motion Controller (LMC) [12][13].…”

“…The experimental results obtained from the previous studies indicate the vision-based system for sign language gestures demonstrates a high accuracy rate with promising results. The vision-based systems such as in [1][2][3][4][5][6][7][8][9][10][11][12][13] also allow the users to engage with more spontaneously and less constraint. However, these approaches have several disadvantages.…”