Hand Gesture Recognition Using Automatic Feature Extraction and Deep Learning Algorithms with Memory

Nogales, Rubén; Benalcázar, Marco E.

doi:10.3390/bdcc7020102

Cited by 14 publications

(9 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further research on SLR is worth doing with the holistic approach. Beneficial implementations can be made with non-typical combinations of DL structures (e.g., attention mechanism, (Bi-)LSTM, (Bi-)LSTM-SVM, (Bi-)RNN) [10,54,55]. Solutions to improve quality of life have inestimable value on society, and it is expected that this work will directly impact the hearing impaired population using PSL on a daily basis in the Republic of Panama.…”

Section: Discussionmentioning

confidence: 99%

Sign-to-Text Translation from Panamanian Sign Language to Spanish in Continuous Capture Mode with Deep Neural Networks

Teran-Quezada,

Lopez-Cabrera,

Rangel

et al. 2024

BDCC

View full text Add to dashboard Cite

Convolutional neural networks (CNN) have provided great advances for the task of sign language recognition (SLR). However, recurrent neural networks (RNN) in the form of long–short-term memory (LSTM) have become a means for providing solutions to problems involving sequential data. This research proposes the development of a sign language translation system that converts Panamanian Sign Language (PSL) signs into text in Spanish using an LSTM model that, among many things, makes it possible to work with non-static signs (as sequential data). The deep learning model presented focuses on action detection, in this case, the execution of the signs. This involves processing in a precise manner the frames in which a sign language gesture is made. The proposal is a holistic solution that considers, in addition to the seeking of the hands of the speaker, the face and pose determinants. These were added due to the fact that when communicating through sign languages, other visual characteristics matter beyond hand gestures. For the training of this system, a data set of 330 videos (of 30 frames each) for five possible classes (different signs considered) was created. The model was tested having an accuracy of 98.8%, making this a valuable base system for effective communication between PSL users and Spanish speakers. In conclusion, this work provides an improvement of the state of the art for PSL–Spanish translation by using the possibilities of translatable signs via deep learning.

show abstract

Section: Discussionmentioning

confidence: 99%

Sign-to-Text Translation from Panamanian Sign Language to Spanish in Continuous Capture Mode with Deep Neural Networks

Teran-Quezada,

Lopez-Cabrera,

Rangel

et al. 2024

BDCC

View full text Add to dashboard Cite

show abstract

“…The methods proposed in [13,14,36] show that purely ML-based classification of radar and accelerometer data is possible for HGR or human motion recognition. Memory-based algorithms introduced in [6] are provided for automated feature extraction for gesture recognition based on camera and infrared sensor data. An amplitude-based model allows the ML model to automatically learn which features in the raw echo data content are relevant for the given classification task.…”

Section: Related Work On Ultrasound-based Systemsmentioning

confidence: 99%

“…Additionally, these technologies should be lean regarding cost and power consumption. There are HGR systems based on other sensors such as vision [5][6][7][8][9][10][11], radar [12][13][14], and WiFi [15][16][17][18][19][20][21][22]. Ultrasound-based HGR systems such as those surveyed in [12] or presented in [23] require the user to wear a device to record the hand gestures.…”

Section: Introductionmentioning

confidence: 99%

End-to-End Ultrasonic Hand Gesture Recognition

Fertl,

Nguyen,

Krueger

et al. 2024

Sensors

View full text Add to dashboard Cite

As the number of electronic gadgets in our daily lives is increasing and most of them require some kind of human interaction, this demands innovative, convenient input methods. There are limitations to state-of-the-art (SotA) ultrasound-based hand gesture recognition (HGR) systems in terms of robustness and accuracy. This research presents a novel machine learning (ML)-based end-to-end solution for hand gesture recognition with low-cost micro-electromechanical (MEMS) system ultrasonic transducers. In contrast to prior methods, our ML model processes the raw echo samples directly instead of using pre-processed data. Consequently, the processing flow presented in this work leaves it to the ML model to extract the important information from the echo data. The success of this approach is demonstrated as follows. Four MEMS ultrasonic transducers are placed in three different geometrical arrangements. For each arrangement, different types of ML models are optimized and benchmarked on datasets acquired with the presented custom hardware (HW): convolutional neural networks (CNNs), gated recurrent units (GRUs), long short-term memory (LSTM), vision transformer (ViT), and cross-attention multi-scale vision transformer (CrossViT). The three last-mentioned ML models reached more than 88% accuracy. The most important innovation described in this research paper is that we were able to demonstrate that little pre-processing is necessary to obtain high accuracy in ultrasonic HGR for several arrangements of cost-effective and low-power MEMS ultrasonic transducer arrays. Even the computationally intensive Fourier transform can be omitted. The presented approach is further compared to HGR systems using other sensor types such as vision, WiFi, radar, and state-of-the-art ultrasound-based HGR systems. Direct processing of the sensor signals by a compact model makes ultrasonic hand gesture recognition a true low-cost and power-efficient input method.

show abstract

“…The characteristic values selected often vary from person to person, and very important characteristic values may be missed. Dimensionality reduction methods such as principal component analysis [30] and deep learning algorithms [31] can extract feature values according to certain rules, so they are expected to solve this problem.…”

Section: Item Symbol Unit Linear Regression Model Regression Model Wi...mentioning

confidence: 99%

Assessing Electrode Characteristics in Continuous Resistance Spot Welding of BH 340 Steel Based on Dynamic Resistance

Zhao,

Vdonin,

Slobodyan

et al. 2023

JMMP

View full text Add to dashboard Cite

The aim of this investigation is to offer a data-based scheme for predicting electrode wear in resistance spot welding. One of the major factors affecting the mechanical properties of spot welds and the variation in weld quality is electrode wear and alloying. In this study, Rogowski coils and twisted pairs attached to the top and bottom electrodes were used to obtain the welding current and the voltage between the electrodes in the welding process, thereby calculating the dynamic resistance value during the welding process. The electrode tip diameter was obtained from the pressure exerted by the upper and lower electrodes on the carbon paper when the current was cut off and was regarded as an indicator of electrode wear. By continuously welding 0.5 mm thick BH 340 steel plates until the electrode failed, the dynamic resistance signal was recorded in real time. Simultaneously, the electrode diameter after every several welds was also recorded. On this basis, the correlation between electrode tip diameter and dynamic resistance is studied. In order to quantitatively study the mapping relationship between dynamic resistance and electrode wear, 10 characteristic values were extracted from the dynamic resistance, and the stepwise regression method was used to obtain the regression formula between the characteristic values and the electrode tip diameter. Using new data to verify the effectiveness of the regression model, the acquired results display that the maximum error between the predicted value of the electrode tip diameter and the measured value obtained by the regression equation with the interactive quadratic term is 0.3 mm, and the corresponding relative error is 7.69%. When welding with a new pair of electrodes, the maximum absolute error was 0.72 mm, and the relative error of the model prediction is within 20% according to the linear regression model with interaction terms. This indicates that this regression model is barely satisfactory for monitoring electrode condition.

show abstract

Hand Gesture Recognition Using Automatic Feature Extraction and Deep Learning Algorithms with Memory

Cited by 14 publications

References 26 publications

Sign-to-Text Translation from Panamanian Sign Language to Spanish in Continuous Capture Mode with Deep Neural Networks

Sign-to-Text Translation from Panamanian Sign Language to Spanish in Continuous Capture Mode with Deep Neural Networks

End-to-End Ultrasonic Hand Gesture Recognition

Assessing Electrode Characteristics in Continuous Resistance Spot Welding of BH 340 Steel Based on Dynamic Resistance

Contact Info

Product

Resources

About