BenSignNet: Bengali Sign Language Alphabet Recognition Using Concatenated Segmentation and Convolutional Neural Network

Abstract

Sign language recognition is one of the most challenging applications in machine learning and human-computer interaction. Many researchers have developed classification models for different sign languages such as English, Arabic, Japanese, and Bengali; however, no significant research has been done on the general-shape performance for different datasets. Most research work has achieved satisfactory performance with a small dataset. These models may fail to replicate the same performance for evaluating different and larger datasets. In this context, this paper proposes a novel method for recognizing Bengali sign language (BSL) alphabets to overcome the issue of generalization. The proposed method has been evaluated with three benchmark datasets such as `38 BdSL’, `KU-BdSL’, and `Ishara-Lipi’. Here, three steps are followed to achieve the goal: segmentation, augmentation, and Convolutional neural network (CNN) based classification. Firstly, a concatenated segmentation approach with YCbCr, HSV and watershed algorithm was designed to accurately identify gesture signs. Secondly, seven image augmentation techniques are selected to increase the training data size without changing the semantic meaning. Finally, the CNN-based model called BenSignNet was applied to extract the features and classify purposes. The performance accuracy of the model achieved 94.00%, 99.60%, and 99.60% for the BdSL Alphabet, KU-BdSL, and Ishara-Lipi datasets, respectively. Experimental findings confirmed that our proposed method achieved a higher recognition rate than the conventional ones and accomplished a generalization property in all datasets for the BSL domain.

Spatial-Temporal Attention with Graph and General Neural Network-based Sign Language Recognition

“…There are many methodologies have been developed to analyse the sign language recognizer over the decades based on computer vision [4,[9][10][11]32]. In the literature many researchers developed sensor-based gesture recognition system such as EEG [33][34][35], hand gloves-based translating approach [36][37][38][39], to reduce the computational cost of the system for real-time implementation of the gesture recognition system.…”

A Sign Language Recognition System Applied to Deaf-Mute Medical Consultation

“…Reducing the resolution of the input image can also improve computational efficiency [ 33 ]. Deep learning is trained through massive data [ 34 ], where the characteristics of the target sign language are automatically learned to complete the detection of the target sign language and the corresponding sign language segmentation is completed through the detected target sign language [ 35 ].…”

Scale-Adaptive Deep Matching Network for Constrained Image Splicing Detection and Localization

“…In recent years, deep learning algorithms [14][15][16] represented by convolutional neural networks [17][18][19], recurrent neural networks [20] and generative adversarial networks have been widely used in many fields such as image classification [21,22], object detection [23], semantic segmentation [24,25], image retrieval [26], scene understanding [27], etc. and have made a leap forward compared with traditional methods.…”