Invisible emotion magnification algorithm (IEMA) for real-time micro-expression recognition with graph-based features

Buhari, Adamu Muhammad; Ooi, Chee‐Pun; Baskaran, Vishnu Monn; Phan, Raphaël C.‐W.; Wong, KokSheik; Tan, Wooi-Haw

doi:10.1007/s11042-021-11625-1

Cited by 8 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Buhari et al proposed a new method to recognize real-time facial emotion. In their studies, the accuracy rate was obtained as 87% in SMIC dataset (Buhari et al, 2022). Although good results have been obtained for the real-time application, it seems that there are results that are open to improvement.…”

Section: Introductionmentioning

confidence: 97%

“…The common features of the approaches can be listed as transforming the multiple classification problem into a binary classification problem, applying the feature selection, and then combining the selected features and using them for classification. For the real time facial expression recognition system, various studies have been done by using deep learning algorithms (Buhari et al, 2022;Umer et al, 2022;Bisogni et al, 2022). Umer et al examined the effects of data augmentation methods to classify facial emotions for different database (Umer et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

“…All these studies indicate the need to increase accuracy for real-time systems in facial expression recognition. It seems that it is difficult to achieve high classification success in real-time systems due to various environmental (light, camera) factors (Buhari et al, 2022;Devrim et al, 2019, Engin et al, 2017. The classification accuracy in this regard needs to be improved.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Sentiment Analysis from Face Expressions Based on Image Processing Using Deep Learning Methods

AKSOY

Güney

2022

Journal of Advanced Research in Natural and Applied Sciences

View full text Add to dashboard Cite

In this study, the classification study of human facial expressions in real-time images is discussed. Implementing this work in software have some benefits for us. For example, analysis of mood in group photos is an interesting instance in this regard. The perception of people’s facial expressions in photographs taken during an event can provide quantitative data on how much fun these people have in general. Another example is context-aware image access, where only photos of people who are surprised can be accessed from a database. Seven different emotions related to facial expressions were classified in this context; these are listed as happiness, sadness, surprise, disgust, anger, fear and neutral. With the application written in Python programming language, classical machine learning methods such as k-Nearest Neighbourhood and Support Vector Machines and deep learning methods such as AlexNet, ResNet, DenseNet, Inception architectures were applied to FER2013, JAFFE and CK+ datasets. In this study, while comparing classical machine learning methods and deep learning architectures, real-time and non-real-time applications were also compared with two different applications. This study conducted to demonstrate that real-time expression recognition systems based on deep learning techniques with the most appropriate architecture can be implemented with high accuracy via computer hardware with only one software. In addition, it is shown that high accuracy rate is achieved in real-time applications when Histograms of Oriented Gradients (HOG) is used as a feature extraction method and ResNet architecture is used for classification.

show abstract

Section: Introductionmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sentiment Analysis from Face Expressions Based on Image Processing Using Deep Learning Methods

AKSOY

Güney

2022

Journal of Advanced Research in Natural and Applied Sciences

View full text Add to dashboard Cite

show abstract

“…To address this problem, [27] proposed a new feature called LBP-FIP which could easily capture dynamic textures from images calculated through five intersecting planes. Similarly, [28] proposed an invisible emotion magnification algorithm (IEMA) which effectively magnifies the strength of facial muscle movement for better classification of microexpression.…”

Section: Introductionmentioning

confidence: 99%

Micro Expression Recognition Using Convolution Patch in Vision Transformer

Indolia,

Nigam,

Singh

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Humans possess an intrinsic ability to hide their true emotions. Micro-expressions are subtle changes in facial muscles that are involuntary by nature and easy to hide. To address these issues, several machine and deep learning models have been proposed in the past few years. Convolution neural network (CNN) is a deep learning method that has widely been adopted in vision-related tasks due to its remarkable performance. However, CNN suffers from overfitting due to a large number of trainable parameters. Additionally, CNN cannot capture global information with respect to an input image. Furthermore, the identification of important regions for the classification of micro-expressions is a challenging task. Selfattention mechanism addresses these issues by focusing on key areas. Furthermore, specific transformers, known as vision transformers are widely explored in vision-related applications. However, existing vision transformers divide an input image into a fixed number of patches due to which local correlation of image pixels is lost. Further, a vision transformer relies on self-attention mechanism which effectively captures global dependencies but does not exploit the local spatial relationships in an image. In this work, we propose a vision transformer based on convolution patches to overcome this problem. The proposed algorithm generates c number of feature maps from input images using c filters through convolution operation. These feature maps are then applied to a transformer model as fixed-size image patches to perform classification. Thus, the proposed architecture leverage advantages of both convolutional layers and transformer, and captures both spatial information and global dependencies respectively, leading to improved performance. The performance of the proposed model is evaluated on three benchmark datasets: CASME-I, CASME-II, and SAMM and compared with state-of-the-art machine and deep learning models, which generated classification accuracy of 95.97%, 98.59%, and 100%, respectively.INDEX TERMS Facial expression recognition, deep learning, micro-expression recognition, self-attention, vision transformer.

show abstract

Ameliorate grasshopper optimization algorithm based long short term memory classification for face emotion recognition system

CH,

2023

Multimed Tools Appl

View full text Add to dashboard Cite

Invisible emotion magnification algorithm (IEMA) for real-time micro-expression recognition with graph-based features

Cited by 8 publications

References 34 publications

Sentiment Analysis from Face Expressions Based on Image Processing Using Deep Learning Methods

Sentiment Analysis from Face Expressions Based on Image Processing Using Deep Learning Methods

Micro Expression Recognition Using Convolution Patch in Vision Transformer

Ameliorate grasshopper optimization algorithm based long short term memory classification for face emotion recognition system

Contact Info

Product

Resources

About