Intelligent Malaysian Sign Language Translation System Using Convolutional‐Based Attention Module with Residual Network

Khan, Rehman Ullah; Khattak, Hizbullah; Wong, Woei Sheng; AlSalman, Hussain; Mosleh, Mogeeb A. A.; Rahman, Sk. Md. Mizanur

doi:10.1155/2021/9023010

Cited by 10 publications

(3 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Adicionalmente, ese mismo año se quintuplicó la cantidad de estudios europeos en comparación con la cantidad de estudios desarrollados el año previo. (43,44,45,46,47) A diferencia de lo reportado, los estudios utilizaron lenguajes de señas para realizar sus intervenciones y el lenguaje de señas indio artículos lo consideran que se adapta bien al momento de interpretar las señas lo que proporciona mayor precisión. (48,49,50) En esta revisión sistemática, se evidencia diferentes objetivos de estudio, los autores propusieron métodos con sensores y otros dispositivos como teléfonos móviles y relojes digitales, lo cual demuestra a diferencia de años anteriores la evolución de los estudios para romper las barreras para las personas con pedida de audición y habla.…”

Section: Discussionunclassified

Using Artificial Intelligence for sign language translation: a systematic literature review

Ortiz-Leon,

Yupanqui-Allcca,

Meneses-Claudio

2023

Salud, Ciencia y Tecnología - Serie de Conferencias

View full text Add to dashboard Cite

Introduction: in this systematic literature review, the use of artificial intelligence in sign language translation for people with hearing and speech loss was analyzed. This review aims to identify the results of the application of artificial intelligence to sign language translation. Method: 462 articles, original and conference papers in SCOPUS, until June 2023, and relying on a selection process based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement, of which 26 studies met exclusion and inclusion criteria. Convolutional Neural Network (CNN) was the most widely implemented machine learning technique in the selected studies. Results: Many sign language systems were tested with various artificial intelligence algorithms and datasets on different continents to create new models and improve sign language translation accuracy. An increasing use of neural networks to achieve better sign language translation efficiency was identified, achieving results ranging from 90 % to 100 % accuracy. Conclusions: The application of artificial intelligence has greatly excelled in the field of Computer Science and has significantly improved the accuracy of sign language translation which has led to lower communication barriers between natural persons and persons with communication disabilities.

show abstract

Section: Discussionunclassified

Using Artificial Intelligence for sign language translation: a systematic literature review

Ortiz-Leon,

Yupanqui-Allcca,

Meneses-Claudio

2023

Salud, Ciencia y Tecnología - Serie de Conferencias

View full text Add to dashboard Cite

show abstract

“…To address this gap, this study proposes a novel approach for diagnosing bearing defects. The approach incorporates wavelet denoising [19] and biphasic current preprocessing, along with the CBAR neural network model adapted from the CBAM-ResNet model [20]. The method aims to enhance the signal quality by applying wavelet denoising to the current signal.…”

Section: Introductionmentioning

confidence: 99%

Bidirectional Current WP and CBAR Neural Network Model-Based Bearing Fault Diagnosis

Zhu,

Liu

2023

IEEE Access

View full text Add to dashboard Cite

In the era of artificial intelligence, the development of an efficient bearing, fault diagnosis method is of vital importance to ensure smooth production operations and avoid major economic losses. To this end, this paper proposes a bearing fault diagnosis method based on biphasic currents. The method first performs wavelet denoising on the biphasic current signal, then extracts its features by simple vector representation and algebraic operations, and finally, combines the CBAR model of Convolutional Block Attention Module (CBAM) and Residual Network (ResNet) for bearing fault diagnosis. The experimental results show that the highest accuracy rate reaches 100% in both single-point fault and single-point mixed with multiple faults conditions on the open source current bearing fault diagnosis dataset, respectively. Compared with other methods, the method proposed in this paper has the advantage of simple data processing, concise model structure, and high-fault diagnosis accuracy, which provides an effective way for dual-phase current-based bearing fault diagnosis. It is worth emphasizing that based on wavelet denoising, this paper uses the simplest vector representation and algebraic operations to preprocess the signal (WP), making the method more efficient and easy to implement.

show abstract

“…The proposed framework is built to be suitable for real-time translation applications between deaf and ordinary people, and to be implemented on mobile devices. According to our previous research on sign language recognition [7,8], this study can be considered a preliminary study on adapting a bidirectional translator to facili-tate bidirectional communication between the deaf and ordinary people. In addition, the study evaluates the performance and efficiency of the different CNN deep learning models within the context of sign language recognition.…”

mentioning

confidence: 99%

A Bidirectional Arabic Sign Language Framework Using Deep Learning and Fuzzy Matching Score

Mosleh,

Assiri,

Gumaei

et al. 2024

Mathematics

Self Cite

View full text Add to dashboard Cite

Sign language is widely used to facilitate the communication process between deaf people and their surrounding environment. Sign language, like most other languages, is considered a complex language which cannot be mastered easily. Thus, technology can be used as an assistive tool to solve the difficulties and challenges that deaf people face during interactions with society. In this study, an automatic bidirectional translation framework for Arabic Sign Language (ArSL) is designed to assist both deaf and ordinary people to communicate and express themselves easily. Two main modules were intended to translate Arabic sign images into text by utilizing different transfer learning models and to translate the input text into Arabic sign images. A prototype was implemented based on the proposed framework by using several pre-trained convolutional neural network (CNN)-based deep learning models, including the DenseNet121, ResNet152, MobileNetV2, Xception, InceptionV3, NASNetLarge, VGG19, and VGG16 models. A fuzzy string matching score method, as a novel concept, was employed to translate the input text from ordinary people into appropriate sign language images. The dataset was constructed with specific criteria to obtain 7030 images for 14 classes captured from both deaf and ordinary people locally. The prototype was developed to conduct the experiments on the collected ArSL dataset using the utilized CNN deep learning models. The experimental results were evaluated using standard measurement metrics such as accuracy, precision, recall, and F1-score. The performance and efficiency of the ArSL prototype were assessed using a test set of an 80:20 splitting procedure, obtaining accuracy results from the highest to the lowest rates with average classification time in seconds for each utilized model, including (VGG16, 98.65%, 72.5), (MobileNetV2, 98.51%, 100.19), (VGG19, 98.22%, 77.16), (DenseNet121, 98.15%, 80.44), (Xception, 96.44%, 72.54), (NASNetLarge, 96.23%, 84.96), (InceptionV3, 94.31%, 76.98), and (ResNet152, 47.23%, 98.51). The fuzzy matching score is mathematically validated by computing the distance between the input and associative dictionary words. The study results showed the prototype’s ability to successfully translate Arabic sign images into Arabic text and vice versa, with the highest accuracy. This study proves the ability to develop a robust and efficient real-time bidirectional ArSL translation system using deep learning models and the fuzzy string matching score method.

show abstract

Intelligent Malaysian Sign Language Translation System Using Convolutional‐Based Attention Module with Residual Network

Cited by 10 publications

References 29 publications

Using Artificial Intelligence for sign language translation: a systematic literature review

Using Artificial Intelligence for sign language translation: a systematic literature review

Bidirectional Current WP and CBAR Neural Network Model-Based Bearing Fault Diagnosis

A Bidirectional Arabic Sign Language Framework Using Deep Learning and Fuzzy Matching Score

Contact Info

Product

Resources

About