Sign Language Translation: A Survey of Approaches and Techniques

Liang, Zeyu; Li, H.; Chai, Jianping

doi:10.3390/electronics12122678

Cited by 16 publications

(4 citation statements)

References 87 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most works are on sign language recognition [3][4][5] which is just to detect the hand gesture and recognize it as an alphabet, number, word or sometimes sentence if it is video, this process of gesture recognition in videos is known as gloss [6]. Ordinarily, gesture recognition of sign language is not always enough in communication as the recognized gloss have to be translated into a readable language text so as to aid communication between less privileged and privileged ones.…”

Section: Print Issn 2714-2469: E-issn 2782-8425 Uniosun Journal Of En...mentioning

confidence: 99%

Development of Mobile Application for Sign Language Translation Using Deep Learning Technique

Omodunbi,

Soladoye,

Okomba

et al. 2024

UJEES

View full text Add to dashboard Cite

Sign Language is a visual and gestural language used by deaf and hard-of-hearing people to communicate. However, communication between hearing and non-hearing individuals can be challenging due to the language barrier. In order to overcome this barrier, a Mobile based Translator is being proposed using Convolutional Neural Network (CNN) to recognize and translate hand gestures in real-time. The proposed system consists of a CNN model trained on a large dataset of hand gestures that includes various signs, such as the alphabet, numbers, and common phrases to recognize various signs, and a backend server to handle the translation of the recognized signs into text. The system was implemented using various CNN architectures like ResNet, MobileNet and VGG-16, where the later gave the best accuracy of 97.69%. The trained VGG-16 model recognizes the signs by extracting features from the images of the hand gestures and using these features to classify the gestures. Once a gesture is recognized, the backend server translates it into text using a pre-defined mapping of signs to words or phrases. The translated text will then be displayed to the user in real-time on the mobile app, enabling seamless communication between hearing and non-hearing individuals. The proposed system was implemented as a mobile app using Flutter, which is a cross-platform development framework. This mobile app make communication easier for vulnerable and enable sharing of information without any discrimination.

show abstract

Section: Print Issn 2714-2469: E-issn 2782-8425 Uniosun Journal Of En...mentioning

confidence: 99%

Development of Mobile Application for Sign Language Translation Using Deep Learning Technique

Omodunbi,

Soladoye,

Okomba

et al. 2024

UJEES

View full text Add to dashboard Cite

show abstract

“…In the literature, the standard hand-communication systems may be broken down into three layers: detection, tracking, and recognition [7]. The task of the detection layer is to identify and extract visual features that may be linked to the presence of hands in the camera's view; for the system to know "what is where" at all times, it is up to the tracking layer to establish temporal data links between successive frames of images.…”

Section: Introductionmentioning

confidence: 99%

“…Signer-free SLR is crucial because it enables the direct application of a technique to the package and the development of the method for signers who were previously unrecognized [8,9]. There are two issues with signerless SLR [7]: There is a noticeable divergence in the signals (postures) of different people, which increases the model convergence complexity. In order to obtain a robust recognition paradigm, the training data must constitute a large number of signers.…”

Section: Introductionmentioning

confidence: 99%

Cognitive Classifier of Hand Gesture Images for Automated Sign Language Recognition: Soft Robot Assistance Based on Neutrosophic Markov Chain Paradigm

Al-Saidi,

Ballagi,

Hassen

et al. 2024

Computers

View full text Add to dashboard Cite

In recent years, Sign Language Recognition (SLR) has become an additional topic of discussion in the human–computer interface (HCI) field. The most significant difficulty confronting SLR recognition is finding algorithms that will scale effectively with a growing vocabulary size and a limited supply of training data for signer-independent applications. Due to its sensitivity to shape information, automated SLR based on hidden Markov models (HMMs) cannot characterize the confusing distributions of the observations in gesture features with sufficiently precise parameters. In order to simulate uncertainty in hypothesis spaces, many scholars provide an extension of the HMMs, utilizing higher-order fuzzy sets to generate interval-type-2 fuzzy HMMs. This expansion is helpful because it brings the uncertainty and fuzziness of conventional HMM mapping under control. The neutrosophic sets are used in this work to deal with indeterminacy in a practical SLR setting. Existing interval-type-2 fuzzy HMMs cannot consider uncertain information that includes indeterminacy. However, the neutrosophic hidden Markov model successfully identifies the best route between states when there is vagueness. This expansion is helpful because it brings the uncertainty and fuzziness of conventional HMM mapping under control. The neutrosophic three membership functions (truth, indeterminate, and falsity grades) provide more layers of autonomy for assessing HMM’s uncertainty. This approach could be helpful for an extensive vocabulary and hence seeks to solve the scalability issue. In addition, it may function independently of the signer, without needing data gloves or any other input devices. The experimental results demonstrate that the neutrosophic HMM is nearly as computationally difficult as the fuzzy HMM but has a similar performance and is more robust to gesture variations.

show abstract

“…Machine understanding of sign languages is a difficult issue and a blooming research field (Liang et al, 2023), with recent advances possible by the availability of benchmark recognition and translation datasets. To create models that generalize effectively, sign language translation and processing require large-scale corpora, much like any other machine translation effort.…”

Section: Introductionmentioning

confidence: 99%

Construção da SILFA

Da Silva,

Kumada,

Costa

et al. 2023

Rev. Incl. e soc.

View full text Add to dashboard Cite

A análise da Língua de Sinais sofre com a falta de acesso a registros de vídeos dos dados bem rotulados, fundamentais para o desenvolvimento dos estudos e adequados para tradução e tecnologias automatizadas. Os conjuntos de dados (com os sinais utilizados no cotidiano da Libras no Brasil) geralmente são limitados a comportamentos criados para estudos específicos e podem diferir em intensidade e tempo do que ocorre espontaneamente. Este artigo apresenta uma descrição detalhada do conjunto de dados de Ação Facial em Língua de Sinais (SILFA) desde da sua concepção até a preparação e análise posterior. Um conjunto de oito jovens surdos adultos foi gravado em vídeo sinalizando sentenças com a intenção de eliciar expressões espontâneas. Além de serem anotadas manualmente de acordo com o sistema de codificação facial, cada quadro de vídeo foi anotado com a presença ou ausência de expressão facial e transcrição em português. O presente estudo visa fornecer informações sobre a presença de expressões faciais em língua de sinais, por meio do protagonismo do Surdo, com destaque no mapeamento de fluxos de vídeos na presença de unidades de ação facial. Além disso, uma análise entre expressões faciais afetivas e gramaticais é exaustivamente discutida. Por fim, o corpus foi criado. Até onde sabemos, este é o primeiro conjunto de dados de língua de sinais brasileira com foco em expressão faciais, em que são fornecidas as frases traduzidas em português escrito, anotações das unidades de ação facial para as capturas de vídeo.

show abstract

Sign Language Translation: A Survey of Approaches and Techniques

Cited by 16 publications

References 87 publications

Development of Mobile Application for Sign Language Translation Using Deep Learning Technique

Development of Mobile Application for Sign Language Translation Using Deep Learning Technique

Cognitive Classifier of Hand Gesture Images for Automated Sign Language Recognition: Soft Robot Assistance Based on Neutrosophic Markov Chain Paradigm

Construção da SILFA

Contact Info

Product

Resources

About