Sign Language Semantic Translation System using Ontology and Deep Learning

Elsayed, Eman K.; Fathy, Doaa

doi:10.14569/ijacsa.2020.0110118

Cited by 31 publications

(19 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performances of the proposed approach with existing state of-the-art techniques on the ArSL2018 dataset in terms of accuracy is shown in Table 8. The findings indicate that the proposed ArSL-CNN model when applying SMOTE resampling to the dataset is superior to two state-of-the-art methods in terms of overall accuracy [1], [3]. Ghazanfar et al [1] used CNN and achieved an accuracy of 95.9%, whereas Elsayed and Fathy [3] applied semantic DL and obtained an accuracy of 88.8%.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 89%

“…Also, increasing the size of the dataset from 33406 samples to 50000 samples resulted in a further increase in the proposed model test accuracy from 94.1% to 95.9%, respectively. Elsayed and Fathy [3] examined the capacity of ontology technologies (semantic web technologies) and DL to design a multiple sign language ontology for feature extraction using CNNs for the ArSL recognition task. Their findings revealed that the recognition rates of the ArSL training and testing sets were 98.06% and 88.87%, respectively.…”

Section: Related Workmentioning

confidence: 99%

“…Among these people, roughly 34 million are under the age of 18. A previous study predicted that this number would double by 2050 due to genetic factors, birth complications, infectious diseases, and chronic ear infections [3], [4]. Studies have been conducted to develop systems that can recognise the signs of various SLs [5].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

ArSL-CNN a convolutional neural network for Arabic sign language gesture recognition

Alani

Cosma

2021

IJEECS

View full text Add to dashboard Cite

<p class="IJASEITAbtract">Sign language (SL) is a visual language means of communication for people who are Deaf or have hearing impairments. In Arabic-speaking countries, there are many Arabic sign languages (ArSL) and these use the same alphabets. This study proposes ArSL-CNN, a deep learning model that is based on a convolutional neural network (CNN) for translating Arabic SL (ArSL). Experiments were performed using a large ArSL dataset (ArSL2018) that contains 54049 images of 32 sign language gestures, collected from forty participants. The results of the first experiments with the ArSL-CNN model returned a train and test accuracy of 98.80% and 96.59%, respectively. The results also revealed the impact of imbalanced data on model accuracy. For the second set of experiments, various re-sampling methods were applied to the dataset. Results revealed that applying the synthetic minority oversampling technique (SMOTE) improved the overall test accuracy from 96.59% to 97.29%, yielding a statistically signicant improvement in test accuracy (p=0.016, α<0=05). The proposed ArSL-CNN model can be trained on a variety of Arabic sign languages and reduce the communication barriers encountered by Deaf communities in Arabic-speaking countries.</p>

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 89%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

ArSL-CNN a convolutional neural network for Arabic sign language gesture recognition

Alani

Cosma

2021

IJEECS

View full text Add to dashboard Cite

show abstract

“…With a score of 99.52%, the suggested system based on residual network ResNet101 obtained the greatest accuracy. Elsayed and Fathy [11] trained and tested Deep CNN architecture on an Arabic sign language dataset. Their experimental results show that the training set's classification accuracy was 98.6%, while the testing sets was 94.31%, according to the collected dataset.…”

Section: Related Workmentioning

confidence: 99%

Static hand gesture recognition of Arabic sign language by using deep CNNs

Ismail

Dawwd

Ali

2021

IJEECS

View full text Add to dashboard Cite

An Arabic sign language recognition using two concatenated deep convolution neural network models DenseNet121 & VGG16 is presented. The pre-trained models are fed with images, and then the system can automatically recognize the Arabic sign language. To evaluate the performance of concatenated two models in the Arabic sign language recognition, the red-green-blue (RGB) images for various static signs are collected in a dataset. The dataset comprises 220,000 images for 44 categories: 32 letters, 11 numbers (0:10), and 1 for none. For each of the static signs, there are 5000 images collected from different volunteers. The pre-trained models were used and trained on prepared Arabic sign language data. These models were used after some modification. Also, an attempt has been made to adopt two models from the previously trained models, where they are trained in parallel deep feature extractions. Then they are combined and prepared for the classification stage. The results demonstrate the comparison between the performance of the single model and multi-model. It appears that most of the multi-model is better in feature extraction and classification than the single models. And also show that when depending on the total number of incorrect recognize sign image in training, validation and testing dataset, the best convolutional neural networks (CNN) model in feature extraction and classification Arabic sign language is the DenseNet121 for a single model using and DenseNet121 & VGG16 for multi-model using.

show abstract

“…The availability of a well-defined system model for carrying out all design tasks, including adjustments and evaluation of the system, is crucial for the system engineers and stakeholders in the acquisition of the system. The use of ontology enables system engineers to not only model metadata concepts but also semantic contexts that can be used in model inference and transformation rulemaking [6]- [8]. Ontology facilitates the process of managing the data obtained because *Corresponding Author www.ijacsa.thesai.org ontology allows the proper arrangement of the entire system [9].…”

Section: Introductionmentioning

confidence: 99%

Transformation of SysML Requirement Diagram into OWL Ontologies

Wardhana¹,

Ashari²,

Kartika³

2020

IJACSA

View full text Add to dashboard Cite

The Requirement Diagrams are used by the System Modeling Language (SysML) to depict and model non-functional requirements, such as response time, size, or system functionality, which cannot be accommodated in the Unified Modeling Language (UML). Nevertheless, SysML still lacks the capability to represent the semantic contexts within the design. Web Ontology Language (OWL) can be used to capture the semantic context of system design; hence, the transformation of SysML diagrams into OWL is needed. The current method of SysML Diagrams transformation into OWL is still done manually so that it is very vulnerable to errors, and the translation process requires more time and effort for system engineers. This research proposes a model that can automatically transform a SysML Requirement Diagram into an OWL file so that system designs can be easily understood by both humans and machines. It also allows users to extract knowledge contained in the previous diagrams. The transformation process makes use of a transformation rule and an algorithm that can be used to change a SysML Requirement Diagram into an OWL ontology file. XML Metadata Interchange (XMI) serialization is used as the bridge to perform the transformation. The produced ontology can be viewed in Protégé. The class and subclass hierarchy, as well as the object properties and data properties, are clearly shown. In the experiment, it is also shown that the model can conduct the transformation correctly.

show abstract

Sign Language Semantic Translation System using Ontology and Deep Learning

Cited by 31 publications

References 13 publications

ArSL-CNN a convolutional neural network for Arabic sign language gesture recognition

ArSL-CNN a convolutional neural network for Arabic sign language gesture recognition

Static hand gesture recognition of Arabic sign language by using deep CNNs

Transformation of SysML Requirement Diagram into OWL Ontologies

Contact Info

Product

Resources

About