Learning Features Robust to Image Variations with Siamese Networks for Facial Expression Recognition

Baddar, Wissam Alhaj; Kim, Dae Hoe; Ro, Yong Man

doi:10.1007/978-3-319-51811-4_16

Cited by 9 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to Reference [22], the signal process of wavelet transform is to select a wavelet family and its transform layers according to the frequency of the input signal, signal granularity, and signal-to-noise ratio to achieve the best effect. With reference [25], we finally determined to apply the db5 wavelet in the Daubechies wavelet family to do 4 layers of multilayer wavelet transform for the data to get the waveform characteristics of the timing signal. In order to obtain the same length as the data collected by the IMU, we extended the transformed sEMG data.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion

Wang

Zhao

Zhou

et al. 2019

Sensors

View full text Add to dashboard Cite

For online sign language recognition (SLR) based on inertial measurement unit (IMU) and a surface electromyography (sEMG) sensor, achieving high-accuracy is a major challenge. The traditional method for that is the segmentation–recognition mechanism, which has two key challenges: (1) it is difficult to design a highly robust segmentation method for online data with inconspicuous segmentation information; and (2) the diversity of input data will increase the burden of the classification. The recognition–verification mechanism was proposed to improve the performance of online SLR. In the recognition stage, we used sliding windows to pull the data, and applied a convolutional neural network (CNN) to classify the sign language signal. In the verification stage, the confidence was evaluated by the Siamese network to judge the correctness of the classification results. The accuracy and rapidity of the classification model were discussed for 86 categories of Chinese sign language. In the experiments for online SLR, the superiority of the recognition–verification mechanism compared to the segmentation–recognition mechanism was verified.

show abstract

Section: Methodsmentioning

confidence: 99%

“…We used the Siamese network based on VGG architecture to verify the classification results, and this model has been widely used in facial expression recognition and object tracking [25,33]. The structure of the Siamese network is shown in Figure 15.…”

Section: Methodsmentioning

confidence: 99%

An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion

Wang

Zhao

Zhou

et al. 2019

Sensors

View full text Add to dashboard Cite

show abstract

“…One of the key features of deep learning is trying to learn the latent features from sample (training) data. However, encoding features that represent all types of variation that could occur in a data sample is hard to achieve (Ding and Tao 2015;Baddar, Kim, and Ro 2017). For example, when considering a population of face images, the face images would have different identities, expressions, poses and illumination variations.…”

Section: Introductionmentioning

confidence: 99%

Mode Variational LSTM Robust to Unseen Modes of Variation: Application to Facial Expression Recognition

Baddar

2019

AAAI

Self Cite

View full text Add to dashboard Cite

Spatio-temporal feature encoding is essential for encoding the dynamics in video sequences. Recurrent neural networks, particularly long short-term memory (LSTM) units, have been popular as an efficient tool for encoding spatio-temporal features in sequences. In this work, we investigate the effect of mode variations on the encoded spatio-temporal features using LSTMs. We show that the LSTM retains information related to the mode variation in the sequence, which is irrelevant to the task at hand (e.g. classification facial expressions). Actually, the LSTM forget mechanism is not robust enough to mode variations and preserves information that could negatively affect the encoded spatio-temporal features. We propose the mode variational LSTM to encode spatio-temporal features robust to unseen modes of variation. The mode variational LSTM modifies the original LSTM structure by adding an additional cell state that focuses on encoding the mode variation in the input sequence. To efficiently regulate what features should be stored in the additional cell state, additional gating functionality is also introduced. The effectiveness of the proposed mode variational LSTM is verified using the facial expression recognition task. Comparative experiments on publicly available datasets verified that the proposed mode variational LSTM outperforms existing methods. Moreover, a new dynamic facial expression dataset with different modes of variation, including various modes like pose and illumination variations, was collected to comprehensively evaluate the proposed mode variational LSTM. Experimental results verified that the proposed mode variational LSTM encodes spatio-temporal features robust to unseen modes of variation.

show abstract

Robust Prostate Cancer Classification with Siamese Neural Networks

Rossi

Bianchini

Scarselli

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Learning Features Robust to Image Variations with Siamese Networks for Facial Expression Recognition

Cited by 9 publications

References 17 publications

An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion

An Recognition–Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion

Mode Variational LSTM Robust to Unseen Modes of Variation: Application to Facial Expression Recognition

Robust Prostate Cancer Classification with Siamese Neural Networks

Contact Info

Product

Resources

About