Persian sign language recognition using IMU and surface EMG sensors

Khomami, Sara Askari; Shamekhi, Sina

doi:10.1016/j.measurement.2020.108471

Cited by 52 publications

(28 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this paper, we focus on the recognition of static hand gestures associated with the letters of the alphabet for fingerspelling, which is part of Task 1. A similar task was also addressed by many other works, such as [19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35], using different approaches for realizing the software and hardware components of the proposed systems.…”

Section: Introductionmentioning

confidence: 86%

See 1 more Smart Citation

Gesture Recognition of Sign Language Alphabet Using a Magnetic Positioning System

et al. 2021

View full text Add to dashboard Cite

Hand gesture recognition is a crucial task for the automated translation of sign language, which enables communication for the deaf. This work proposes the usage of a magnetic positioning system for recognizing the static gestures associated with the sign language alphabet. In particular, a magnetic positioning system, which is comprised of several wearable transmitting nodes, measures the 3D position and orientation of the fingers within an operating volume of about 30 × 30 × 30 cm, where receiving nodes are placed at known positions. Measured position data are then processed by a machine learning classification algorithm. The proposed system and classification method are validated by experimental tests. Results show that the proposed approach has good generalization properties and provides a classification accuracy of approximately 97% on 24 alphabet letters. Thus, the feasibility of the proposed gesture recognition system for the task of automated translation of the sign language alphabet for fingerspelling is proven.

show abstract

Section: Introductionmentioning

confidence: 86%

“…Sensor-based gesture recognition systems are commonly implemented as data gloves featuring inertial measurement units and stretch sensors such as in [30][31][32], as surface electromyography sensors [33], or other wearable devices [34,35]. Sensor-based approaches have the key advantage of simplifying the detection process.…”

Section: Introductionmentioning

confidence: 99%

Gesture Recognition of Sign Language Alphabet Using a Magnetic Positioning System

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Namun, teknik berbasis computer-vision memerlukan pencahayaan yang konstan dan kompleks (Zhang, dkk, 2019) dan (Gupta & Kumar, 2021), dan penempatan kamera yang sesuai. Teknik ini juga mahal untuk diimplementasikan (Khomami & Shamekhi, 2021), (Wu, dkk, 2016). Dalam penelitian ini memilih teknik berbasis sensor MPU-6050 yang terdiri dari accelerometer 3-sumbu, dan 3-sumbu gyroscope untuk mengukur percepatan gerakan, dan kecepatan sudut (Abdullah, dkk, 2020).…”

Section: Pendahuluanunclassified

“…Dalam penelitian ini memilih teknik berbasis sensor MPU-6050 yang terdiri dari accelerometer 3-sumbu, dan 3-sumbu gyroscope untuk mengukur percepatan gerakan, dan kecepatan sudut (Abdullah, dkk, 2020). Selain itu, sensor IMU dipilih karena biaya yang murah, konsumsi daya rendah, dan dapat digunakan untuk melacak gerakan tangan secara keseluruhan (Khomami & Shamekhi, 2021) (Filippeschi, dkk, 2017).…”

Section: Pendahuluanunclassified

Pengenalan Pola Dasar Angka berdasarkan Gerakan Tangan menggunakan Machine Learning

2022

View full text Add to dashboard Cite

ABSTRAKPengenalan gerakan tangan dianggap sebagai bagian penting dari interaksi manusia komputer, memungkinkan komputer untuk mengenali dan menafsirkan gerakan tangan dan menjalankan perintah. Penggunaan machine learning dimanfaatkan untuk mencari tren dan pola yang berbeda. Namun, tantangan untuk menerapkan machine learning menjadi bagaimana memilih di antara berbagai model berbeda digunakan untuk kumpulan data atau kasus berbeda. Tujuan dari penelitian ini adalah mengukur kinerja model machine learning yang diusulkan dengan pemilihan hyperparameter yang sesuai dalam mengenali 10 pola angka berdasarkan gerakan tangan di udara. Dalam makalah ini, model KNN, SVM, dan ANN-PSO diusulkan. Eksperimen dilakukan dengan mengumpulkan data gerakan yang berasal dari MPU-6050. Kinerja metode yang diusulkan dievaluasi menggunakan metrik standar seperti akurasi klasifikasi, presisi, recall, f1-score, dan AUC-ROC. Hasilnya menunjukkan bahwa akurasi rata-rata mencapai 87%.Kata kunci: HCI, hand gesture recognition, machine learning, MPU-6050, pola ABSTRACTHand gesture recognition is considered an essential part of human-computer interaction (HCI), enabling computers to recognize and interpret hand gesturesand execute commands. The use of machine learning is utilized to look for different trends and patterns. However, the challenge for implementing machine learning becomes how to choose between different models used for different datasets or cases. This research aims to measure the performance of the proposed machine learning model by selecting the appropriate hyperparameters in recognizing 10 number patterns based on hand gestures in the air. In this paper, KNN, SVM, and ANN-PSO models are proposed. Experiments were carried by collecting gesture data from MPU-6050. The performance of the proposed method was evaluated using standard metrics such as classification accuracy, precision, recall, f1-score, and AUC-ROC. The results show that the average accuracy reaches 87%.Keywords: HCI, hand gesture recognition, machine learning, MPU-6050, pattern

show abstract

“…Non-negative matrix factorization is used to extract synergistic effects from a subset of training poses and used to predict the EMG patterns of the remaining poses. Khomami et al [48] designed and produced a wearable low-cost device based on surface EMG and inertial measurement unit (IMU) sensors. The fusion of the two sensors will improve the accuracy of the system's capture of signals.…”

mentioning

confidence: 99%

Neural Decoding of Chinese Sign Language With Machine Learning for Brain–Computer Interfaces

Wang¹,

Zhou

et al. 2021

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Limb motion decoding is an important part of brain-computer interface (BCI) research. Among the limb motion, sign language not only contains rich semantic information and abundant maneuverable actions but also provides different executable commands. However, many researchers focus on decoding the gross motor skills, such as the decoding of ordinary motor imagery or simple upper limb movements. Here we explored the neural features and decoding of Chinese sign language from electroencephalograph (EEG) signal with motor imagery and motor execution. Sign language not only contains rich semantic information, but also has abundant maneuverable actions, and provides us with more different executable commands. In this paper, twenty subjects were instructed to perform movement execution and movement imagery based on Chinese sign language. Seven classifiers are employed to classify the selected features of sign language EEG. L1 regularization is used to learn and select features that contain more information from the mean, power spectral density, sample entropy, and brain network connectivity. The best average classification accuracy of the classifier is 89.90% (imagery sign language is 83.40%). These results have shown the feasibility of decoding between different sign languages. The source location reveals that the neural circuits involved in sign language are related to the visual contact area and the pre-movement area. Experimental evaluation shows that the proposed decoding strategy based on sign language can obtain outstanding classification results, which provides a certain reference value for the subsequent research of limb decoding based on sign language.

show abstract

Persian sign language recognition using IMU and surface EMG sensors

Cited by 52 publications

References 19 publications

Gesture Recognition of Sign Language Alphabet Using a Magnetic Positioning System

Gesture Recognition of Sign Language Alphabet Using a Magnetic Positioning System

Pengenalan Pola Dasar Angka berdasarkan Gerakan Tangan menggunakan Machine Learning

Neural Decoding of Chinese Sign Language With Machine Learning for Brain–Computer Interfaces

Contact Info

Product

Resources

About