Multi-Modal Pain Intensity Recognition Based on the SenseEmotion Database

Abstract

sential roles in my academic journey while completing my Ph.D. dissertation. Their support, mentorship, and presence have been invaluable, and I am deeply appreciative. I must begin by acknowledging the support of my family: Edalat, Parivash, and Milad. Their belief in my aspirations and constant encouragement has strengthened me throughout this demanding journey.I am profoundly grateful to my distinguished Ph.D. committee, particularly Professor Friedhelm Schwenker, to whom I owe special thanks for his mentorship.

Exploring Deep Physiological Models for Nociceptive Pain Recognition

“…Similar performances have been achieved in the field of 25 speech recognition [8,9] and natural language processing [10,11]. 26 A steadily growing amount of work has been exploring the application of deep 27 learning approaches on physiological signals. Martinéz et al [12] were able to 28 significantly outperform standard approaches built upon hand-crafted features by using 29 a deep learning algorithm for affect modelling based on physiological signals (two 30 physiological signals consisting of Skin Conductance (SC) and Blood Volume Pulse 31 (BVP) were used in this specific work).…”

“…Finally, 154 a third order bandpass Butterworth filter with a frequency range of [0.1, 250] Hz was 155 applied on the ECG signal. Furthermore, the data is segmented as proposed in [33], but 156 rather than using 5.5 sec windows with a shift of 3 sec from the elicitations' onset, the 157 preprocessed signals were segmented into windows of length 4.5 sec, with a shift from 158 the elicitations' onset of 4 sec (see Fig 2(a)) based on the data driven signal 159 segmentation approach that was recently proposed in [27]. Each signal extracted within 160 this window constitutes a 1-D array of size 4.5 × 256 = 1152 and is later on used in 161 combination with the corresponding level of pain elicitation to optimize and assess the 162 designed deep classification architectures.…”

“…63pain intensity classification models, since most of the previous works on pain intensity 64 classification involving autonomic parameters rely on a carefully designed set of 65 hand-crafted features. Most recently, Thiam et al[27] provided the results for a row of 66 pain intensity classification experiments based on the SenseEmotion Database 67 (SEDB)[28], by using several fusion architectures to merge hand-crafted features 68 extracted from different input modalities, including physiological, audio and video 69 channels. Thereby, the combination of the features extracted from the recorded signals 70 was compared for different fusion approaches, i.e.…”

Two-Stream Attention Network for Pain Recognition from Video Sequences

“…Recent advances in both domains of computer vision and machine learning, combined with the release of several datasets designed specifically for pain-related research (e.g., UNBC-McMaster Shouder Pain Expression Archive Database [3], BioVid Heat Pain Database [4], Multimodal EmoPain Database [5] and SenseEmotion Database [6]), have fostered the development of a multitude of automatic pain assessment and classification approaches. These methods range from unimodal approaches, characterised by the optimisation of an inference model based on one unique and specific input signal (e.g., video sequences [7,8], audio signals [9,10] and bio-physiological signals [11][12][13]), to multimodal approaches that are characterised by the optimisation of an information fusion architecture based on parameters stemming from a set of distinctive input signals [14][15][16].…”

“…Those descriptors are further used to perform the classification of several levels of pain intensities using a Random Forest (RF) [32] model. Similarly, the authors in [7,14,15,33], propose several spatio-temporal descriptors extracted either from the localised facial area or from the estimated head pose, including, among others, Pyramid Histograms of Oriented Gradients (PHOG) [34] and Local Gabor Binary Patterns from Three Orthogonal Planes (LGBP-TOP) [35], to perform the classification of several levels of nociceptive pain. The classification experiments are also performed with RF models and ANNs.…”