An Effective Approach to Communicate with the Deaf and Mute People by Recognizing Characters of One-hand Bangla Sign Language Using Convolutional Neural-Network

Rony, Alfat Jahan; Saikat, Khairul Hossain; Tanzeem, Mahdia; Robi, F.M. Rahat Hasan

doi:10.1109/ceeict.2018.8628158

Cited by 8 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A dual purpose interpreter of one handed Bangla sign language that can do both text to sign language and sign language to text conversion was proposed by Rony, Saikat, Tanzeem, and Robi (2018). They used the idea of transfer learning and used InceptionV3 as the backbone CNN model.…”

Section: Bangla Sign Language (Bsl)mentioning

confidence: 99%

A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier

Das

Imtiaz²,

Neom³

et al. 2023

Expert Systems with Applications

View full text Add to dashboard Cite

Section: Bangla Sign Language (Bsl)mentioning

confidence: 99%

A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier

Das

Imtiaz²,

Neom³

et al. 2023

Expert Systems with Applications

View full text Add to dashboard Cite

“…Penyandang tunarungu mengandalkan bahasa isyarat untuk berkomunikasi dengan manusia lain (Golîmba, 2022). Bahasa isyarat adalah salah satu media komunikasi terbaik untuk orang Tuli dan Bisu yang tidak dapat berbicara dengan orang lain atau mendengar dari orang lain (Rony 2018). Penyandang tunarungu dan bisu menggunakan bahasa isyarat sebagai sarana utama mereka untuk mengungkapkan pikiran dan gagasan mereka kepada orang-orang di sekitar mereka dengan gerakan tangan dan tubuh yang berbeda (Boppana, 2019).…”

Section: Pendahuluanunclassified

Untitled

Sarinastiti¹,

Susanto²,

Astin³

et al. 2022

idealog

View full text Add to dashboard Cite

Abstrak : Tunarungu merupakan pengalaman berkurangnya kemampuan mendengarkan yang membuat orang tidak dapat menangkap rangsangan melalui indera pendengar. Ketunarunguan akan membawa dampak pada pendidikan yang diperoleh penderitanya. Saat berkomunikasi, bahasa yang digunakan dalam kesehariannya yaitu bahasa isyarat. Salah satu provinsi, yang memiliki penyandang tunarungu terbanyak di Indonesia adalah di Jawa Timur. TK SLB Tunarungu Karya Mulia Surabaya merupakan salah satu sekolah yang mengkhususkan untuk penderita tunarungu. TK ini memiliki media pembelajaran berupa rolling picture yang berfungsi secara manual. Hal tersebut membuat proses pembelajaran menjadi kurang efisien. Dengan itu perlu adanya pengembangan rolling picture yang dapat digerakkan secara otomatis menggunakan remot. Rolling picture otomatis berisikan materi sepanjang 9 Meter dengan berat 4.160 gram dan dapat dikendalikan dengan remote control menggunakan dua tombol. Saat tombol pertama (previous) ditekan, maka motor akan bergerak CW (Clockwise) yaitu searah jarum jam, untuk menjalankan materi selanjutnya, dan tombol (next) maka motor akan bergerak CCW (Counterclockwise) yaitu berlawanan dengan arah jarum jam untuk mengubah pada materi sebelumnya. Penggunaan remote control dapat mengendalikan alat dengan maksimal jarak sejauh 5 meter. Diharapkan dengan adanya media pembelajaran ini dapat memudahkan guru untuk mengajar serta mempermudah siswa dalam mempelajari kosakata bahasa isyarat. Kata kunci : tunarungu, media pembelajaran, desain rolling picture otomatis, desain poster kosakata Abstract : Deafness is an experience of reduced listening ability that makes people unable to perceive stimuli through the sense of hearing. Deafness will have an impact on the education received by the sufferer. When communicating, the language used in daily life is sign language. One of the provinces with the most deaf people in Indonesia is East Java.. SLB Tunarungu Karya Mulia kindergarten of Surabaya is one of the schools specializing in hearing impaired people. This kindergarten has a learning media in the form of Rolling Picture that works manually. This makes the learning process less efficient. With this, it is necessary to develop a Rolling Picture that can be moved automatically using a remote. Automatic Rolling Picture contains 9 meters long material with a weight of 4,160 grams and can be controlled with remote using two buttons. When the first button (previous) is pressed, the motor will move CW (Clockwise) that is clockwise, to run the next material, and the button (next) then the motor will move CCW (Counterclockwise) which is counterclockwise to change the previous material. The use of remote can control the tool with a maximum distance of 5 meters. It is expected that this learning media can facilitate teachers to teach and facilitate students in learning sign language vocabulary.

show abstract

“…Many researchers applied CNN to recognize BdSL. Islam et al [19], Hoque et al [20], and Islam et al [21] utilized CNN to recognize two-handed BdSL alphabets, whereas, Rony et al [22], Hossen et al [23], and Rafi et al [24] implemented CNN to recognize one-handed…”

Section: Literature Reviewmentioning

confidence: 99%

Design and Development of a Humanoid Robot for Sign Language Interpretation

et al. 2021

View full text Add to dashboard Cite

Purpose With an indisputable complexity of communication for hearing and speaking impaired people, most sign language recognition systems utilize virtual reality or onscreen robots. This paper presents the design and development of a special and low-cost humanoid robot that can perform as a sign language interpreter. To the best of our knowledge, this is the first endeavor to fabricate a humanoid robot for Bangla sign language (BdSL) and Medical signs interpretation. Methods Considering the plethora of design criteria and balancing between rigidity and flexibility 3D models of the robotics parts are designed and 3D printed ensuring cost efficiency. With the help of modern fabrication technology, the robot is developed and assembled with proper actuators and circuitry. An image dataset is built comprising 950 images for BdSL recognition and made publicly available. We utilized the Recurrent neural network (RNN) and Convolutional neural network (CNN) for deep learning model establishment and feature extraction from video and image data. Results The developed humanoid robot has 43 Degrees of freedom (DoF) which includes two 15 DoF hands. It can imitate 16 BdSL alphabets in sign language, can capture a video or image input in real-time from the user, and recognize 10 medical signs and 38 alphabets of BdSL. The learning model for video-based medical sign recognition achieved 87.5% test accuracy. Image-based Bangla sign language recognition achieved an overall test accuracy of 98.19% in our dataset and 93.8% in another available dataset. Conclusion Compared to the state-of-the-art robotic systems for sign language interpretation, our approach has achieved higher kinematic characteristics, remarkable results in sign recognition, and impressive competency in sign imitation; all at almost 10 times lower cost than the state-of-the-art systems. The results are evidence that our approach is efficient and suitable in helping hearing and speaking impaired people. Moreover, this work initiates a research scope that can be further extended for creating equal opportunities for the hearing and speaking impaired community.

show abstract

An Effective Approach to Communicate with the Deaf and Mute People by Recognizing Characters of One-hand Bangla Sign Language Using Convolutional Neural-Network

Cited by 8 publications

References 7 publications

A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier

A hybrid approach for Bangla sign language recognition using deep transfer learning model with random forest classifier

Untitled

Design and Development of a Humanoid Robot for Sign Language Interpretation

Contact Info

Product

Resources

About