Mexican Sign Language Corpus: Towards an Automatic Translator

Trujillo-Romero, Felipe; García-Bautista, Gibran

doi:10.1145/3591471

Cited by 4 publications

(3 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many types of SLs have been used in previous studies based on countries and spoken languages. Languages used in SL research are numerous including American [16,22,[41][42][43][44][45][46], Mexican [47], Arabic [3,7,10,11,39,40,48,49], Algerian [50], Pakistani [51], Indian [52][53][54], Bangla [55], Chinese [56,57], Indonesian [58], Italian [23], Peruvian [59], Turkish [34], and Urdu [24]. Based on [60], which analyzed SL classification research studies published from 2014 to 2021, another breakdown of literature works was presented according to the local variation in the SL they referred to.…”

Section: Overview Of Sl Classification Research Based On Languagesmentioning

confidence: 99%

“…One significant challenge for SL research is the limited data availability, particularly for ArSL [33]. Datasets, also called databases in earlier research on SL, can be obtained through public or private repositories [16,23,47,59]. Datasets of SL can be divided into three main categories: finger spelling, isolated signs, and continuous signs.…”

Section: Overview Of Sl Classification Research Based On Datasetsmentioning

confidence: 99%

See 1 more Smart Citation

Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition

Alharthi,

Alzahrani

2023

Applied Sciences

View full text Add to dashboard Cite

Sign languages are complex, but there are ongoing research efforts in engineering and data science to recognize, understand, and utilize them in real-time applications. Arabic sign language recognition (ArSL) has been examined and applied using various traditional and intelligent methods. However, there have been limited attempts to enhance this process by utilizing pretrained models and large-sized vision transformers designed for image classification tasks. This study aimed to create robust transfer learning models trained on a dataset of 54,049 images depicting 32 alphabets from an ArSL dataset. The goal was to accurately classify these images into their corresponding Arabic alphabets. This study included two methodological parts. The first one was the transfer learning approach, wherein we utilized various pretrained models namely MobileNet, Xception, Inception, InceptionResNet, DenseNet, and BiT, and two vision transformers namely ViT, and Swin. We evaluated different variants from base-sized to large-sized pretrained models and vision transformers with weights initialized from the ImageNet dataset or otherwise randomly. The second part was the deep learning approach using convolutional neural networks (CNNs), wherein several CNN architectures were trained from scratch to be compared with the transfer learning approach. The proposed methods were evaluated using the accuracy, AUC, precision, recall, F1 and loss metrics. The transfer learning approach consistently performed well on the ArSL dataset and outperformed other CNN models. ResNet and InceptionResNet obtained a comparably high performance of 98%. By combining the concepts of transformer-based architecture and pretraining, ViT and Swin leveraged the strengths of both architectures and reduced the number of parameters required for training, making them more efficient and stable than other models and existing studies for ArSL classification. This demonstrates the effectiveness and robustness of using transfer learning with vision transformers for sign language recognition for other low-resourced languages.

show abstract

Section: Overview Of Sl Classification Research Based On Languagesmentioning

confidence: 99%

Section: Overview Of Sl Classification Research Based On Datasetsmentioning

confidence: 99%

Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition

Alharthi,

Alzahrani

2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Garcia and Viesca [8], focused on classifying alphabet handshape correctly for letters a-k instead of all types of ASL alphabets. Another work presented in [23] detected ASL signs and converted to audio, and authors of [24] focused on constructing a corpus using the Mexican Sign Language (MSL).…”

Section: Literature Reviewmentioning

confidence: 99%

An AI-Based Framework for Translating American Sign Language to English and Vice Versa

Avina,

Amiruzzaman,

Amiruzzaman

et al. 2023

Information

View full text Add to dashboard Cite

In this paper, we propose a framework to convert American Sign Language (ASL) to English and English to ASL. Within this framework, we use a deep learning model along with the rolling average prediction that captures image frames from videos and classifies the signs from the image frames. The classified frames are then used to construct ASL words and sentences to support people with hearing impairments. We also use the same deep learning model to capture signs from the people with deaf symptoms and convert them into ASL words and English sentences. Based on this framework, we developed a web-based tool to use in real-life application and we also present the tool as a proof of concept. With the evaluation, we found that the deep learning model converts the image signs into ASL words and sentences with high accuracy. The tool was also found to be very useful for people with hearing impairment and deaf symptoms. The main contribution of this work is the design of a system to convert ASL to English and vice versa.

show abstract

LSC50: Colombian Sign Language Video and Inertial Measurement dataset

Felipe Flórez-Sierra,

David Solórzano,

Segura-Quijano

et al. 2024

Sci Data

View full text Add to dashboard Cite

In Colombia, approximately more than 500.000 people have disabling hearing loss, representing around 1% of the population in Colombia, and only 400 professional interpreters of Colombian Sign Language (LSC) are available in the whole country. In this data descriptor, we present a dataset with recordings of Colombian Sign Language (LSC) as an important effort to develop technologies that facilitate social inclusion and equity for the deaf community in Colombia. This dataset includes multimodal recordings of native and non-native signers signing different elements of the LSC lexicon. In particular, we provide recordings of video and inertial measurement units, along with detected body landmarks and computed information on the inverse kinematics of the upper body such as flexion, extension, and rotation of arms and wrists. This dataset is the first of its kind for LSC and it is key for the development of technologies needed to better understand the language and reduce communication barriers between people with and without hearing loss.

show abstract

Mexican Sign Language Corpus: Towards an Automatic Translator

Cited by 4 publications

References 42 publications

Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition

Vision Transformers and Transfer Learning Approaches for Arabic Sign Language Recognition

An AI-Based Framework for Translating American Sign Language to English and Vice Versa

LSC50: Colombian Sign Language Video and Inertial Measurement dataset

Contact Info

Product

Resources

About