ASL Recognition Based on Kinematics Derived from a Multi-Frequency RF Sensor Network

Gürbüz, Sevgi Zübeyde; Gürbüz, Ali Cafer; Malaia, Evie; Griffin, Darrin J.; Crawford, Chris; Kurtoğlu, Emre; Rahman, Mohammad Mahbubur; Aksu, Ridvan; Mdrafi, Robiulhossain

doi:10.1109/sensors47125.2020.9278864

“…Literature [7] demonstrated that radar data can capture linguistic features of sign language for distinguishing between everyday interactions and sign language. Literature [8] extracted 4 handcrafted features from radar data in three different frequency bands and used machine learning to identify 20 American Sign Languages. Literature [9] studied 4 British Sign Language classifications using deep learning methods.…”

Section: Introductionmentioning

confidence: 99%

Chinese sign language recognition based on multi-view deep neural network for millimeter-wave radar

Wang

¹

,

Cui

²

,

Liu

³

et al. 2023

Conference on Infrared, Millimeter, Terahertz Waves and Applications (IMT2022)

0

View full text Add to dashboard Cite

People in the deaf-mute community benefit a lot from Chinese sign language (CSL) recognition, which can promote communication between sign language users and non-users. Recently, some studies have been made on sign language recognition with the millimeter-wave radar because of its advantages of non-contact measurements and privacy controls. The millimeter-wave radar acquires the motion characteristics based on the micro-Doppler images, which can be used for CSL recognition. Existing recognition methods measure the micro-Doppler image in a certain direction, which cannot reflect all the motion information of CSL and leads to the failure of recognition of the CSL with similar actions. In order to improve the recognition accuracy, this paper proposes a multi-view deep neural network (MV-DNN), which fuses micro-Doppler features measured in different directions. The simulation results show that the recognition accuracy of the proposed method reaches 96% for eight CSLs, which is 8% higher than that of the traditional single-view method.

show abstract

“…Literature [7] demonstrated that radar data can capture linguistic features of sign language for distinguishing between everyday interactions and sign language. Literature [8] extracted 4 handcrafted features from radar data in three different frequency bands and used machine learning to identify 20 American Sign Languages. Literature [9] studied 4 British Sign Language classifications using deep learning methods.…”

Section: Introductionmentioning

confidence: 99%

Chinese sign language recognition based on multiview deep neural network for millimeter wave radar

Wang

¹

,

Cui²,

Liu³

et al. 2022

Artificial Intelligence and Machine Learning in Defense Applications IV

2

0

View full text Add to dashboard Cite

People in the deaf-mute community benefit a lot from Chinese sign language (CSL) recognition, which can promote communication between sign language users and non-users. Recently, some studies have been made on sign language recognition with the millimeter-wave radar because of its advantages of non-contact measurements and privacy controls. The millimeter-wave radar acquires the motion characteristics based on the micro-Doppler images, which can be used for CSL recognition. Existing recognition methods measure the micro-Doppler image in a certain direction, which cannot reflect all the motion information of CSL and leads to the failure of recognition of the CSL with similar actions. In order to improve the recognition accuracy, this paper proposes a multi-view deep neural network (MV-DNN), which fuses micro-Doppler features measured in different directions. The simulation results show that the recognition accuracy of the proposed method reaches 96% for eight CSLs, which is 8% higher than that of the traditional single-view method.

show abstract

A Survey: The Sensor-Based Method for Sign Language Recognition

Yang,

Shen,

Wang

et al. 2023

Lecture Notes in Computer Science

0

View full text Add to dashboard Cite

ASL Recognition Based on Kinematics Derived from a Multi-Frequency RF Sensor Network

Cited by 6 publications

References 18 publications

Chinese sign language recognition based on multi-view deep neural network for millimeter-wave radar

Chinese sign language recognition based on multi-view deep neural network for millimeter-wave radar

Chinese sign language recognition based on multiview deep neural network for millimeter wave radar

A Survey: The Sensor-Based Method for Sign Language Recognition

Contact Info

Product

Resources

About