An Intelligent Human–Unmanned Aerial Vehicle Interaction Approach in Real Time Based on Machine Learning Using Wearable Gloves

Müezzinoğlu, Taha; Karaköse, Mehmet

doi:10.3390/s21051766

Cited by 27 publications

(9 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we conducted a test to confirm the classification performance of static gesture recognition. We used confusion matrix as metric to usefully express the classification performance of the static gestures by referring to the preliminary research on the HGR system [ 15 , 16 , 25 , 36 ]. Figure 12 shows the confusion matrix and normalized confusion matrix on the test set, and the diagonal value of the normalized confusion matrix shows high density.…”

Section: Methodsmentioning

confidence: 99%

“…ML-based SGR approach is a method of using machine-learning technology, and there are various classification techniques for gesture recognition algorithms, namely, decision tree (DT) [ 20 ], artificial neural networks (ANN) [ 20 , 21 ], K-nearest neighbors (KNN) [ 22 ], and SVM [ 23 , 24 ]. In Ref [ 25 ], Muezzinoglu, T. compared the results extracted from data gloves for DT, SVM, and KNN classification algorithms. Although ML-based approaches have the advantage of high accuracy, they have the disadvantage of high computation and poor accuracy for data for people who do not undergo training.…”

Section: Previous Researchmentioning

confidence: 99%

See 1 more Smart Citation

Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time

Yoo

Song

et al. 2022

Sensors

View full text Add to dashboard Cite

As an alternative to traditional remote controller, research on vision-based hand gesture recognition is being actively conducted in the field of interaction between human and unmanned aerial vehicle (UAV). However, vision-based gesture system has a challenging problem in recognizing the motion of dynamic gesture because it is difficult to estimate the pose of multi-dimensional hand gestures in 2D images. This leads to complex algorithms, including tracking in addition to detection, to recognize dynamic gestures, but they are not suitable for human–UAV interaction (HUI) systems that require safe design with high real-time performance. Therefore, in this paper, we propose a hybrid hand gesture system that combines an inertial measurement unit (IMU)-based motion capture system and a vision-based gesture system to increase real-time performance. First, IMU-based commands and vision-based commands are divided according to whether drone operation commands are continuously input. Second, IMU-based control commands are intuitively mapped to allow the UAV to move in the same direction by utilizing estimated orientation sensed by a thumb-mounted micro-IMU, and vision-based control commands are mapped with hand’s appearance through real-time object detection. The proposed system is verified in a simulation environment through efficiency evaluation with dynamic gestures of the existing vision-based system in addition to usability comparison with traditional joystick controller conducted for applicants with no experience in manipulation. As a result, it proves that it is a safer and more intuitive HUI design with a 0.089 ms processing speed and average lap time that takes about 19 s less than the joystick controller. In other words, it shows that it is viable as an alternative to existing HUI.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Previous Researchmentioning

confidence: 99%

Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time

Yoo

Song

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…ANOVA was used to assess signal samples, and the results were 86.1 percent successful. Muezzinoglu et al [8] used Gsr and IMU sensors to collect data from eight people-two women and six men-with 4-10 years of driving experience who used the glove built for the study. The authors used the data to create a stress classifier model that uses an SVM to distinguish between stressful and non-stressful driving circumstances.…”

Section: A Glove Made Of the Internet Of Things (Iot) Based On Deep L...mentioning

confidence: 99%

A New Intelligent Approach for Deaf/Dumb People based on Deep Learning

Elwahsh¹,

Elkhouly²,

Nasr³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

People who are deaf or have difficulty speaking use sign language, which consists of hand gestures with particular motions that symbolize the "language" they are communicating. A gesture in a sign language is a particular movement of the hands with a specific shape from the fingers and whole hand. In this paper, we present an Intelligent for Deaf/Dumb People approach in real time based on Deep Learning using Gloves (IDLG). The approach IDLG offers scientific contributions based deep-learning, a multimode command techniques, real-time, and effective use, and high accuracy rates. For this purpose, smart gloves working in real time were designed. The data obtained from the gloves was processed using deep-learning-based approaches and classified multi-mode commands that allow dumb people to speak with regular people via their smart phone. Internally, the glove has five flex sensors and an accelerometer using to achieve Low-Cost Control System. The flex sensor generates a proportional change in resistance for each individual move. The processing of these hand gestures is in Atmega32A Microcontroller which is an advance version of the microcontroller and the lab view software. IDLG compares the input signal to memory-stored specified voltage values. The performance of the IDLG approach was verified on a dataset created using different hand gestures from 20 different people. In the test using the IDLG approach on 10,000 data points, process time performance of milliseconds was achieved with 97% accuracy.

show abstract

“…For example, ref. [18] presents an intelligent human-UAV interaction approach in real time based on machine learning using wearable gloves; it used data gloves in the Multi-Mode UAV Human-Computer Interaction System.…”

Section: Introductionmentioning

confidence: 99%

“…At present, commonly used algorithms include sensor fusion algorithm [19], closed-form reconstruction algorithm [11], supervised learning algorithm [15], etc. Muezzinoglu and Karakose [18] compared decision tree, Nave Bayes, support vector machines, and k-nearest neighbor classification methods in the interaction between data gloves and UAV. Finally, experiments are needed to verify the control effect of the algorithm results of the data glove on the UAV.…”

Section: Introductionmentioning

confidence: 99%

Virtual interaction and manipulation control of a hexacopter through hand gesture recognition from a data glove

Huang

Liang

et al. 2022

Robotica

View full text Add to dashboard Cite

The purpose of this study is to realize virtual interaction and manipulation control of a hexacopter based on hand gesture recognition from a designed data glove, to provide an intuitive and visual real-time simulation system for flight control algorithm verification and external control equipment testing. First, the hand gesture recognition from a designed data glove is studied, which can recognize different actions, such as mobile ready, grab, loosen, landing, take-off, and hover. Then, the design of virtual simulation system for hexacopter capture is completed, with the model design of hexacopter and manipulator, and the simulation software design with $CoppeliaSim$ . Finally, virtual simulation experiment of hexacopter grasping and virtual flight control experiment based on data glove are tested, respectively, and quantitatively described. The overall recognition rate is 84.3%, indicating that the data glove produced has the ability to recognize gestures, but its recognition performance is not superior. In gesture recognition, the recognition rate of static gestures is relatively higher than that of dynamic gestures. Among the static gestures, the hover gesture has the highest recognition rate. The average correct rate of static gestures can reach 94%. The lowest recognition rate of dynamic gestures is upward movement, and the average recognition rate of dynamic gestures is 76.1%. The research can be used to remotely operate hexacopter using a data glove in the future and improve the control performance through virtual interaction and manipulation simulation before actual application.

show abstract

An Intelligent Human–Unmanned Aerial Vehicle Interaction Approach in Real Time Based on Machine Learning Using Wearable Gloves

Cited by 27 publications

References 49 publications

Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time

Motion Estimation and Hand Gesture Recognition-Based Human–UAV Interaction Approach in Real Time

A New Intelligent Approach for Deaf/Dumb People based on Deep Learning

Virtual interaction and manipulation control of a hexacopter through hand gesture recognition from a data glove

Contact Info

Product

Resources

About