A Modified CNN Network for Automatic Pain Identification Using Facial Expressions

Abstract

Pain is a strong symptom of diseases. Being an involuntary unpleasant feeling, it can be considered a reliable indicator of health issues. Pain has always been expressed verbally, but in some cases, traditional patient self-reporting is not efficient. On one side, there are patients who have neurological disorders and cannot express themselves accurately, as well as patients who suddenly lose consciousness due to an abrupt faintness. On another side, medical staff working in crowded hospitals need to focus on emergencies and would opt for the automation of the task of looking after hospitalized patients during their entire stay, in order to notice any pain-related emergency. These issues can be tackled with deep learning. Knowing that pain is generally followed by spontaneous facial behaviors, facial expressions can be used as a substitute to verbal reporting, to express pain. In this paper, a convolutional neural network (CNN) model was built and trained to detect pain through patients' facial expressions, using the UNBC-McMaster Shoulder Pain dataset. First, faces were detected from images using the Haarcascade Frontal Face Detector provided by OpenCV, and preprocessed through gray scaling, histogram equalization, face detection, image cropping, mean filtering, and normalization. Next, preprocessed images were fed into a CNN model which was built based on a modified version of the VGG16 architecture. The model was finally evaluated and fine-tuned in a continuous way based on its accuracy, which reached 92.5%.

Facial Expressions as Behavioral Indicators for Assessing Pain using Machine Learning Models

“…IV. Models examined are a modified VGG16 model, a recurrent neural network (RNN) and VGG16, and hybrid frameworks for facial video using a combination of CNN and RNN (Karamitsos et al 2021) (Bargshady et al 2019) (Bellantonio et al 2017). A high accuracy of 99% for pain level predictions were obtained when two pretrained CNNs used either VGG16, InceptionV3 or ResNet50, and a shallow CNN (Alghamdi et al 2022).…”

Pain Detection From Facial Expressions Based on Transformers and Distillation

“…This database consists of 20 adults with stimulated electrical pain. In a recent work by Karamitsos et al [15], the authors proposed a novel Convolutional Neural Network (CNN) for automatic pain detection from facial expressions. The proposed CNN consists of a modified version of VGG16 [16] model.…”

Patient Monitoring Using Emotion Recognition

“…According to Karamitsos and others [6], grayscale keeps one channel for each image and simplifies the convolution operation. Color is not effective in this matter.…”