Multi-Modal Pain Intensity Assessment Based on Physiological Signals: A Deep Learning Perspective

Abstract

Traditional pain assessment approaches ranging from self-reporting methods, to observational scales, rely on the ability of an individual to accurately assess and successfully report observed or experienced pain episodes. Automatic pain assessment tools are therefore more than desirable in cases where this specific ability is negatively affected by various psycho-physiological dispositions, as well as distinct physical traits such as in the case of professional athletes, who usually have a higher pain tolerance as regular individuals. Hence, several approaches have been proposed during the past decades for the implementation of an autonomous and effective pain assessment system. These approaches range from more conventional supervised and semi-supervised learning techniques applied on a set of carefully hand-designed feature representations, to deep neural networks applied on preprocessed signals. Some of the most prominent advantages of deep neural networks are the ability to automatically learn relevant features, as well as the inherent adaptability of trained deep neural networks to related inference tasks. Yet, some significant drawbacks such as requiring large amounts of data to train deep models and over-fitting remain. Both of these problems are especially relevant in pain intensity assessment, where labeled data is scarce and generalization is of utmost importance. In the following work we address these shortcomings by introducing several novel multi-modal deep learning approaches (characterized by specific supervised, as well as self-supervised learning techniques) for the assessment of pain intensity based on measurable bio-physiological data. While the proposed supervised deep learning approach is able to attain state-of-the-art inference performances, our self-supervised approach is able to significantly improve the data efficiency of the proposed architecture by automatically generating physiological data and simultaneously performing a fine-tuning of the architecture, which has been previously trained on a significantly smaller amount of data.

Tree-Based Models for Pain Detection from Biomedical Signals

“…They applied a bidirectional Long Short-Term Memory (LSTM) network to learn temporal dynamic characteristics of physiological signals and fused them with handcrafted features. Thiam et al [25] proposed a multi-modal information aggregation approach based on Deep Denoising Convolutional Auto-Encoders for pain assessment on two different pain databases including BioVid Heat Pain Database. Thaim et al [26] designed Convolutional Neural Network (CNN) based on physiological signals (EDA, ECG, and EMG) for pain classification.…”

“…New features that we computed are (22) the number of R peaks in the windows of 5.5 s (Fig. 1a); (23) the range of R amplitudes; (24) the standard deviation of R amplitudes; (25) the mean of the duration of PT (Fig 1b ):…”

An Automatic System for Continuous Pain Intensity Monitoring Based on Analyzing Data from Uni-, Bi-, and Multi-Modality

“…Most methods of pain recognition used a single modality [44,50] used video, [51,52] used audio signal, and [10,12,33] used physiological signals. The recent methods used multiple modalities [3,4,14,16,[53][54][55] that can improve the performance and flexibility of pain recognition. Thiam et al [54] proposed multi-modal methods to develop pain intensity classification.…”

“…The recent methods used multiple modalities [3,4,14,16,[53][54][55] that can improve the performance and flexibility of pain recognition. Thiam et al [54] proposed multi-modal methods to develop pain intensity classification. They introduced a supervised deep learning method and a selfsupervised method for recognizing pain intensity based on physiological signals.…”

Machine Learning-Based Pain Intensity Estimation: Where Pattern Recognition Meets Chaos Theory—An Example Based on the BioVid Heat Pain Database

“…Moreover, the implementation of welladvanced models, such as (deep) artificial neural networks (ANNs) [15], does not sufficiently close the gap to the ground truth (e.g. [21] and [23]), as for instance in comparison to several image-based classification tasks including one or even two hundred classes [1] (e.g. defined by the CIFAR-100 [13] or Caltech Birds [29] data sets).…”