A Structured and Methodological Review on Vision-Based Hand Gesture Recognition System

Farid, Fahmid Al; Hashim, Noramiza; Abdullah, Junaidi; Bhuiyan, Md Roman; Mohd-Isa, Wan-Noorshahida; Uddin, Jia; Haque, Mohammad A.; Husen, Mohd Nizam

doi:10.3390/jimaging8060153

Cited by 49 publications

(15 citation statements)

References 92 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, these technologies should be lean regarding cost and power consumption. There are HGR systems based on other sensors such as vision [5][6][7][8][9][10][11], radar [12][13][14], and WiFi [15][16][17][18][19][20][21][22]. Ultrasound-based HGR systems such as those surveyed in [12] or presented in [23] require the user to wear a device to record the hand gestures.…”

Section: Introductionmentioning

confidence: 99%

End-to-End Ultrasonic Hand Gesture Recognition

Fertl,

Nguyen,

Krueger

et al. 2024

Sensors

View full text Add to dashboard Cite

As the number of electronic gadgets in our daily lives is increasing and most of them require some kind of human interaction, this demands innovative, convenient input methods. There are limitations to state-of-the-art (SotA) ultrasound-based hand gesture recognition (HGR) systems in terms of robustness and accuracy. This research presents a novel machine learning (ML)-based end-to-end solution for hand gesture recognition with low-cost micro-electromechanical (MEMS) system ultrasonic transducers. In contrast to prior methods, our ML model processes the raw echo samples directly instead of using pre-processed data. Consequently, the processing flow presented in this work leaves it to the ML model to extract the important information from the echo data. The success of this approach is demonstrated as follows. Four MEMS ultrasonic transducers are placed in three different geometrical arrangements. For each arrangement, different types of ML models are optimized and benchmarked on datasets acquired with the presented custom hardware (HW): convolutional neural networks (CNNs), gated recurrent units (GRUs), long short-term memory (LSTM), vision transformer (ViT), and cross-attention multi-scale vision transformer (CrossViT). The three last-mentioned ML models reached more than 88% accuracy. The most important innovation described in this research paper is that we were able to demonstrate that little pre-processing is necessary to obtain high accuracy in ultrasonic HGR for several arrangements of cost-effective and low-power MEMS ultrasonic transducer arrays. Even the computationally intensive Fourier transform can be omitted. The presented approach is further compared to HGR systems using other sensor types such as vision, WiFi, radar, and state-of-the-art ultrasound-based HGR systems. Direct processing of the sensor signals by a compact model makes ultrasonic hand gesture recognition a true low-cost and power-efficient input method.

show abstract

Section: Introductionmentioning

confidence: 99%

End-to-End Ultrasonic Hand Gesture Recognition

Fertl,

Nguyen,

Krueger

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Raimundo F et al and Roman et al provide a convolutional neural network-based approach for Hajj applications [4], [5], [7], [10]. Some other different deep learning-based approaches are also considered in the current trend of gesture recognition arena [6], [8], [9], [11], [12].…”

Section: Introductionmentioning

confidence: 99%

“…In terms of accuracy, our suggested methodology outperforms the best available techniques. 4. We have created our own gesture dataset on Different Camera Orientations (DCOG)…”

Section: Introductionmentioning

confidence: 99%

Single Shot Detector CNN and Deep Dilated Masks for Vision-Based Hand Gesture Recognition From Video Sequences

Farid,

Hashim,

Abdullah

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

With an increasing number of people on the planet today, innovative human-computer interaction technologies and approaches may be employed to assist individuals in leading more fulfilling lives. Gesture-based technology has the potential to improve the safety and well-being of impaired people, as well as the general population. Recognizing gestures from video streams is a difficult problem because of the large degree of variation in the characteristics of each motion across individuals. In this article, we propose applying deep learning methods to recognize automated hand gestures using RGB and depth data. To train neural networks to detect hand gestures, any of these forms of data may be utilized. Gesturebased interfaces are more natural, intuitive, and straightforward. Earlier study attempted to characterize hand motions in a number of contexts. Our technique is evaluated using a vision-based gesture recognition system. In our suggested technique, image collection starts with RGB video and depth information captured with the Kinect sensor and is followed by tracking the hand using a single shot detector Convolutional Neural Network (SSD-CNN). When the kernel is applied, it creates an output value at each of the m × n locations. Using a collection of convolutional filters, each new feature layer generates a defined set of gesture detection predictions. After that, we perform deep dilation to make the gesture in the image masks more visible. Finally, hand gestures have been detected using the well-known classification technique SVM. Using deep learning we recognize hand gestures with higher accuracy of 93.68% in RGB passage, 83.45% in the depth passage, and 90.61% in RGB-D conjunction on the SKIG dataset compared to the state-of-theart. In the context of our own created Different Camera Orientation Gesture (DCOG) dataset we got higher accuracy of 92.78% in RGB passage, 79.55% in the depth passage, and 88.56% in RGB-D conjunction for the gestures collected in 0-degree angle. Moreover, the framework intends to use unique methodologies to construct a superior vision-based hand gesture recognition system.

show abstract

“…Farmers with a lack of technical competence and agricultural professionals are unable to serve the field at all times, causing them to overlook the most effective preventative opportunities. In recent times, image processing [1], pattern recognition [2,3], computer vision [4,5] and text [6][7][8][9] or video [6,10] analysis have fast advanced outcomes in recent years. A machine learning-based intelligent disease detection approach provides a means for effectively resolving agricultural concerns [11,12].…”

Section: Introductionmentioning

confidence: 99%

Multimodal Hybrid Deep Learning Approach to Detect Tomato Leaf Disease Using Attention Based Dilated Convolution Feature Extractor with Logistic Regression Classification

Islam

Sultana

Farid

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Automatic leaf disease detection techniques are effective for reducing the time-consuming effort of monitoring large crop farms and early identification of disease symptoms of plant leaves. Although crop tomatoes are seen to be susceptible to a variety of diseases that can reduce the production of the crop. In recent years, advanced deep learning methods show successful applications for plant disease detection based on observed symptoms on leaves. However, these methods have some limitations. This study proposed a high-performance tomato leaf disease detection approach, namely attention-based dilated CNN logistic regression (ADCLR). Firstly, we develop a new feature extraction method using attention-based dilated CNN to extract most relevant features in a faster time. In our preprocessing, we use Bilateral filtering to handle larger features to make the image smoother and the Ostu image segmentation process to remove noise in a fast and simple way. In this proposed method, we preprocess the image with bilateral filtering and Otsu segmentation. Then, we use the Conditional Generative Adversarial Network (CGAN) model to generate a synthetic image from the image which is preprocessed in the previous stage. The synthetic image is generated to handle imbalance and noisy or wrongly labeled data to obtain good prediction results. Then, the extracted features are normalized to lower the dimensionality. Finally, extracted features from preprocessed data are combined and then classified using fast and simple logistic regression (LR) classifier. The experimental outcomes show the state-of-the-art performance on the Plant Village database of tomato leaf disease by achieving 100%, 100%, 96.6% training, testing, and validation accuracy, respectively, for multiclass. From the experimental analysis, it is clearly demonstrated that the proposed multimodal approach can be utilized to detect tomato leaf disease precisely, simply and quickly. We have a potential plan to improve the model to make it cloud-based automated leaf disease classification for different plants.

show abstract

A Structured and Methodological Review on Vision-Based Hand Gesture Recognition System

Cited by 49 publications

References 92 publications

End-to-End Ultrasonic Hand Gesture Recognition

End-to-End Ultrasonic Hand Gesture Recognition

Single Shot Detector CNN and Deep Dilated Masks for Vision-Based Hand Gesture Recognition From Video Sequences

Multimodal Hybrid Deep Learning Approach to Detect Tomato Leaf Disease Using Attention Based Dilated Convolution Feature Extractor with Logistic Regression Classification

Contact Info

Product

Resources

About