Accuracy is not enough: a heterogeneous ensemble model versus FGSM attack

Abstract

In this paper, based on facial landmark approaches, the possible vulnerability of ensemble algorithms to the FGSM attack has been assessed using three commonly used models: convolutional neural network-based antialiasing (A_CNN), Xc_Deep2-based DeepLab v2, and SqueezeNet (Squ_Net)-based Fire modules. Firstly, the three individual deep learning classifier-based Facial Emotion Recognition (FER) classifications have been developed; the predictions from all three classifiers are then merged using majority voting to develop the HEM_Net-based ensemble model. Following that, an in-depth investigation of their performance in the case of attack-free has been carried out in terms of the Jaccard coefficient, accuracy, precision, recall, F1 score, and specificity. When applied to three benchmark datasets, the ensemble-based method (HEM_Net) significantly outperforms in terms of precision and reliability while also decreasing the dimensionality of the input data, with an accuracy of 99.3%, 87%, and 99% for the Extended Cohn-Kanade (CK+), Real-world Affective Face (RafD), and Japanese female facial expressions (Jaffee) data, respectively. Further, a comprehensive analysis of the drop in performance of every model affected by the FGSM attack is carried out over a range of epsilon values (the perturbation parameter). The results from the experiments show that the advised HEM_Net model accuracy declined drastically by 59.72% for CK + data, 42.53% for RafD images, and 48.49% for the Jaffee dataset when the perturbation increased from A to E (attack levels). This demonstrated that a successful Fast Gradient Sign Method (FGSM) can significantly reduce the prediction performance of all individual classifiers with an increase in attack levels. However, due to the majority voting, the proposed HEM_Net model could improve its robustness against FGSM attacks, indicating that the ensemble can lessen deception by FGSM adversarial instances. This generally holds even as the perturbation level of the FGSM attack increases.