Human facial neural activities and gesture recognition for machine-interfacing applications

Hamedi, Mahyar; Salleh, Sheikh Hussain Shaikh; Swee, Tan Tian; Ismail, Kamarul; Ali, Jafar; Dee-Uam, C.; Pavaganun, C.; Yupapin, Preecha P.

doi:10.2147/ijn.s26619

Cited by 14 publications

(12 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It must be mentioned that, comparing the overall performance of the previous works with the results of this paper was not fair since the number of classes as well as the participants, signal recording protocol and the considered facial gestures were not the same. When comparing with [23] in which a similar setup was considered, it should be noticed that despite the lower accuracy (about 3%) achieved by VEBFNN, this classifier was considerably faster than FCM.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

EMG-based facial gesture recognition through versatile elliptic basis function neural network

Hamedi

Salleh

Astaraki

et al. 2013

BioMedical Engineering OnLine

View full text Add to dashboard Cite

BackgroundRecently, the recognition of different facial gestures using facial neuromuscular activities has been proposed for human machine interfacing applications. Facial electromyograms (EMGs) analysis is a complicated field in biomedical signal processing where accuracy and low computational cost are significant concerns. In this paper, a very fast versatile elliptic basis function neural network (VEBFNN) was proposed to classify different facial gestures. The effectiveness of different facial EMG time-domain features was also explored to introduce the most discriminating.MethodsIn this study, EMGs of ten facial gestures were recorded from ten subjects using three pairs of surface electrodes in a bi-polar configuration. The signals were filtered and segmented into distinct portions prior to feature extraction. Ten different time-domain features, namely, Integrated EMG, Mean Absolute Value, Mean Absolute Value Slope, Maximum Peak Value, Root Mean Square, Simple Square Integral, Variance, Mean Value, Wave Length, and Sign Slope Changes were extracted from the EMGs. The statistical relationships between these features were investigated by Mutual Information measure. Then, the feature combinations including two to ten single features were formed based on the feature rankings appointed by Minimum-Redundancy-Maximum-Relevance (MRMR) and Recognition Accuracy (RA) criteria. In the last step, VEBFNN was employed to classify the facial gestures. The effectiveness of single features as well as the feature sets on the system performance was examined by considering the two major metrics, recognition accuracy and training time. Finally, the proposed classifier was assessed and compared with conventional methods support vector machines and multilayer perceptron neural network.ResultsThe average classification results showed that the best performance for recognizing facial gestures among all single/multi-features was achieved by Maximum Peak Value with 87.1% accuracy. Moreover, the results proved a very fast procedure since the training time during classification via VEBFNN was 0.105 seconds. It was also indicated that MRMR was not a proper criterion to be used for making more effective feature sets in comparison with RA.ConclusionsThis work was accomplished by introducing the most discriminating facial EMG time-domain feature for the recognition of different facial gestures; and suggesting VEBFNN as a promising method in EMG-based facial gesture classification to be used for designing interfaces in human machine interaction systems.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Since there is not much work reported on facial EMG analysis, this paper considers the same setup used in [23] to investigate more on the impact of different facial EMG features on the classification of facial gestures. Therefore, characteristics of ten facial gestures EMGs were explored by extracting ten different time-domain features.…”

Section: Introductionmentioning

confidence: 99%

EMG-based facial gesture recognition through versatile elliptic basis function neural network

Hamedi

Salleh

Astaraki

et al. 2013

BioMedical Engineering OnLine

View full text Add to dashboard Cite

show abstract

“…Their general aim has been to categorise phonemes, words, articulatory features, or gestures by facial and tongue EMG signals [7–14]. Honda et al [15] and Lucero & Munhall [16] have both published on predicting lip shapes.…”

Section: Introductionmentioning

confidence: 99%

Predicting 3D lip shapes using facial surface EMG

et al. 2017

View full text Add to dashboard Cite

AimThe aim of this study is to prove that facial surface electromyography (sEMG) conveys sufficient information to predict 3D lip shapes. High sEMG predictive accuracy implies we could train a neural control model for activation of biomechanical models by simultaneously recording sEMG signals and their associated motions.Materials and methodsWith a stereo camera set-up, we recorded 3D lip shapes and simultaneously performed sEMG measurements of the facial muscles, applying principal component analysis (PCA) and a modified general regression neural network (GRNN) to link the sEMG measurements to 3D lip shapes. To test reproducibility, we conducted our experiment on five volunteers, evaluating several sEMG features and window lengths in unipolar and bipolar configurations in search of the optimal settings for facial sEMG.ConclusionsThe errors of the two methods were comparable. We managed to predict 3D lip shapes with a mean accuracy of 2.76 mm when using the PCA method and 2.78 mm when using modified GRNN. Whereas performance improved with shorter window lengths, feature type and configuration had little influence.

show abstract

“…For each analysis window of each movement pattern ( Mi, i = 1, 2, …, 5, listed in Table 1 but excluding “no movement”), the root mean square (RMS) amplitude (24, 25) and the 4th-order autoregressive (AR) model coefficients (18) of each channel were calculated as features. One of the AR model coefficients that is constant was removed, so that there was F = 5 features from each channel each analysis window.…”

Section: Methodsmentioning

confidence: 99%

Hands-Free Human-Computer Interface Based on Facial Myoelectric Pattern Recognition

Zhou

2019

Front. Neurol.

View full text Add to dashboard Cite

Patients with no or limited hand function usually have difficulty in using conventional input devices such as a mouse or a touch screen. Having the ability of manipulating electronic devices can give patients full access to the digital world, thereby increasing their independence and confidence, and enriching their lives. In this study, a hands-free human-computer interface was developed in order to help patients manipulate computers using facial movements. Five facial movement patterns were detected by four electromyography (EMG) sensors, and classified using myoelectric pattern recognition algorithms. Facial movement patterns were mapped to cursor actions including movements in different directions and click. A typing task and a drawing task were designed in order to assess the interaction performance of the interface in daily use. Ten able-bodied subjects participated in the experiment. In the typing task, the median path efficiency was 80.4%, and the median input rate was 5.9 letters per minute. In the drawing task, the median time to accomplish was 239.9 s. Moreover, all the subjects achieved high classification accuracy (median: 98.0%). The interface driven by facial EMG achieved high performance, and will be assessed on patients with limited hand functions in the future.

show abstract

Human facial neural activities and gesture recognition for machine-interfacing applications

Cited by 14 publications

References 30 publications

EMG-based facial gesture recognition through versatile elliptic basis function neural network

EMG-based facial gesture recognition through versatile elliptic basis function neural network

Predicting 3D lip shapes using facial surface EMG

Hands-Free Human-Computer Interface Based on Facial Myoelectric Pattern Recognition

Contact Info

Product

Resources

About