A proposed artificial intelligence-based real-time speech-to-text to sign language translator for South African official languages for the COVID-19 era and beyond: In pursuit of solutions for the hearing impaired

Abstract

Background: The emergence of the coronavirus disease 2019 (COVID-19) pandemic has resulted in communication being heightened as one of the critical aspects in the implementation of interventions. Delays in the relaying of vital information by policymakers have the potential to be detrimental, especially for the hearing impaired.Objectives: This study aims to conduct a scoping review on the application of artificial intelligence (AI) for real-time speech-to-text to sign language translation and consequently propose an AI-based real-time translation solution for South African languages from speech-to-text to sign language.Methods: Electronic bibliographic databases including ScienceDirect, PubMed, Scopus, MEDLINE and ProQuest were searched to identify peer-reviewed publications published in English between 2019 and 2021 that provided evidence on AI-based real-time speech-to-text to sign language translation as a solution for the hearing impaired. This review was done as a precursor to the proposed real-time South African translator.Results: The review revealed a dearth of evidence on the adoption and/or maximisation of AI and machine learning (ML) as possible solutions for the hearing impaired. There is a clear lag in clinical utilisation and investigation of these technological advances, particularly in the African continent.Conclusion: Assistive technology that caters specifically for the South African community is essential to ensuring a two-way communication between individuals who can hear clearly and individuals with hearing impairments, thus the proposed solution presented in this article.

Improving communication and promoting social inclusion for hearing-impaired users: Usability evaluation and design recommendations for assistive mobile applications

“…Recent studies have critically assessed the limitations and social challenges associated with conventional cochlear implants and hearing aids developed using intelligent assistive technologies [ 12 , 13 ]. A segment of this research indicates that the adoption of such technologies is often hindered by societal factors [ 14 ].…”

Ten Years of TeleHealth and Digital Healthcare: Where Are We?

“…Furthermore, even in TH and DH, one can currently count on the use of emerging technologies, such as robotics [ 24 , 25 , 26 , 27 ], augmented reality [ 28 ], virtual reality [ 29 ], and artificial intelligence [ 30 , 31 ]. Moreover, assistive technologies have played a strategic role during the pandemic as they were consolidated in the context of different applications and disabilities relating communication and motion [ 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 ].…”

Robot Assist Sign Language Recognition for Hearing Impaired Persons Using Deep Learning

“…The results demonstrate that the CNN technique delivers up to 86% accuracy, while the Support Vector Trained model achieves an accuracy of 84% among machine learning algorithms [48]. The purpose of this research [49] is to perform a scoping review of the utilization of AI in instant translation applications from spoken language to sign language. The goal is to provide an AI-based technique to turn South African Sign languages from monologue-to-text to sign language, allowing for interactive interaction between those who can hear and those who cannot.…”