Real-Time Hand Gesture Communication System in Hindi for Speech and Hearing Impaired

Chaman, Shilpa; D'souza, Dylan; D'mello, Benz; Bhavsar, Karan; D'souza, Tolton

doi:10.1109/iccons.2018.8663015

Cited by 13 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shilpa et al [9] demonstrated a real-time model to identify the Hindi language based gestures. The author also developed an application to convert the gesture to text (G2T) and text to gesture (T2G) processes.…”

Section: Literature Reviewmentioning

confidence: 99%

Precise Recognition of Vision Based Multi-hand Signs Using Deep Single Stage Convolutional Neural Network

Bose

Kumar

2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

The precise recognition of multi-hand signs in real-time under dynamic backgrounds, illumination conditions is a time consuming process. In this paper, a time efficient single stage convolutional neural network (CNN) You Only Look Once (YOLO-V2) model is proposed for real-time multi-hand sign recognition. The model utilizes DarkNet-19 CNN architecture as a feature extractor. The model is trained and tested on three distinct datasets (NUSHP-II, SENZ-3D and MITI-HD). The range of IoU from 0.5 to 0.95, the model is validated using test dataset. For the MITI-HD, the YOLO-V2 CNN model obtained an average precision value of 99.10% for AP 0.5 ; 93.00% for AP 0.75 and 78.30% for AP 0.5:0.95 . The Adam Optimizer on YOLO-V2 CNN model supersedes the other optimization methods. The prediction time of YOLO-V2 CNN is obtained as 20 ms, much lower than other single-stage hand sign recognition systems.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Precise Recognition of Vision Based Multi-hand Signs Using Deep Single Stage Convolutional Neural Network

Bose

Kumar

2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

show abstract

“…www.ijew.io People with hearing disability live a compromised life, to improve the standard of their lives a lot of different techniques can be implemented, one of them is the conversion of gesture to text and vice versa. A method to convert gesture to text and text to gesture for Hindi Language has been developed, the images of palm gestures were given as an input to extract the region of hand, based on skin color segmentation [11]. A 5-bit binary string to extract features was used to identify 32 different gestures.…”

Section: Literature Reviewmentioning

confidence: 99%

Hindko Sign Language (HSL) Recognition Using Convolutional Neural Network

Raza¹,

Ullah²

2020

IJEW

View full text Add to dashboard Cite

Language has always played a significant role in Human to Human communication. In case of not knowing someone else's language, one can use hand gestures for communicating crudely but still be able to convey the message. Other than not knowing someone else's language there are millions of people in the world who have hearing or speaking disability. According to the World Health Organization (WHO), it has been estimated to be a population of 436 million (5% of the total world's population) people in the world who have hearing disabilities. Deaf people cannot use oral languages for communicating with other people. The source of their communication is Sign Language (SL) that conveys the message to the other person. In Computer Vision, there are different algorithms, which are used to interpret gestures and recognize them. The Deaf community of Pakistan uses its SL, like any other country in the world i.e., Pakistan Sign Language (PSL). There are around 60 local languages that are spoken in Pakistan including Hindko. Hindko and many other local languages spoken by minorities in Pakistan are on the brink of being endangered as the amount of research done on these languages is almost negligible. In this paper Convolutional Neural Network (CNN) is used for the recognition of Hindko Sign Language (HSL). Furthermore, we examine and analyze the recognition based on prediction to evaluate the efficiency of the utilized CNN. The methodology developed in this research work achieved an accuracy rate of 99.98%.

show abstract

“…Another research use a webcam to recognize visual motion [7] The application of hand gestures in addition to computers is also carried out for navigation controllers on drones [8]. Hand gesture recognition is also implemented to translate Indonesian [9], Hindi [10] and American sign language [11] [12].…”

Section: Introductionmentioning

confidence: 99%

Prototype Program Hand Gesture Recognize Using the Convex Hull Method and Convexity Defect on Android

Anshary

Hidayat

Amalia

2020

join

View full text Add to dashboard Cite

One of the research topics of Human-Computer Interaction is the development of input devices and how users interact with computers. So far, the application of hand gestures is more often applied to desktop computers. Meanwhile, current technological developments have given rise to various forms of computers, one of which is a computer in the form of a smartphone whose users are increasing every year. Therefore, hand gestures need to be applied to smartphones to facilitate interaction between the user and the device. This study implements hand gestures on smartphones using the Android operating system. The algorithm used is convex hull and convexity defect for recognition of the network on the hand which is used as system input. Meanwhile, to ensure this technology runs well, testing was carried out with 3 scenarios involving variable lighting, background color, and indoor or outdoor conditions. The results of this study indicate that Hand gesture recognition using convex hull and convexity defect algorithms has been successfully implemented on smartphones with the Android operating system. Indoor or outdoor testing environment greatly affects the accuracy of hand gesture recognition. For outdoor use, a green background color with a light intensity of 1725 lux produces 76.7% accuracy, while for indoors, a red background color with a light intensity of 300 lux provides the greatest accuracy of 83.3%.

show abstract

Real-Time Hand Gesture Communication System in Hindi for Speech and Hearing Impaired

Cited by 13 publications

References 16 publications

Precise Recognition of Vision Based Multi-hand Signs Using Deep Single Stage Convolutional Neural Network

Precise Recognition of Vision Based Multi-hand Signs Using Deep Single Stage Convolutional Neural Network

Hindko Sign Language (HSL) Recognition Using Convolutional Neural Network

Prototype Program Hand Gesture Recognize Using the Convex Hull Method and Convexity Defect on Android

Contact Info

Product

Resources

About