A Spatiotemporal Convolutional Neural Network for Automatic Pain Intensity Estimation from Facial Dynamics

Tavakolian, Mohammad; Hadid, Abdenour

doi:10.1007/s11263-019-01191-3

Cited by 50 publications

(31 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, Zu et al [69] concluded that 33 of chest CTs can have rounded lung opacities. Machine learning (ML) techniques are attracting substantial interest in the medical field, where deep learning-based models have been successfully utilized in many healthcare applications such as depression detection [36], pain estimation [54], breast cancer detection [10], Alzheimer's disease classification [18], and pneumonia detection from chest X-ray images [2]. Due to the increase in COVID-19 cases, healthcare systems have been overwhelmed and require alternative solutions for the automated diagnosis of COVID-19.…”

Section: Introductionmentioning

confidence: 99%

SAM: Self-augmentation mechanism for COVID-19 detection using chest X-ray Images

Usman¹,

Hoque²,

Oussalah³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p>COVID-19 is a rapidly spreading viral disease and has affected over 100 countries worldwide. The numbers of casualties and infected cases have been escalated particularly in vulnerable states with weakened healthcare systems. Recently, reverse transcription-polymerase chain reaction (RT-PCR) is the test of choice for diagnosing COVID-19. However, current evidence suggests that COVID-19 infected patients are mostly stimulated from a lung infection after coming in contact with this virus. Therefore, chest X-ray (i.e., radiography) and chest CT can be a surrogate in some countries where PCR is not readily available. This has forced the scientific community to detect COVID-19 infection from X-ray images and recently proposed machine learning methods offer great promise for fast and accurate detection. Deep learning with convolutional neural networks (CNNs) has been successfully applied to radiological imaging for improving the accuracy of diagnosis. However, the performance remains limited due to the lack of representative X-ray images available in public benchmark datasets. To alleviate this issue, we propose an attention mechanism for data augmentation in the feature space rather than in the data space using reconstruction independent component analysis (RICA). Specifically, a unified architecture is proposed which contains a deep convolutional neural network (CNN), an attention mechanism, and a bidirectional LSTM (BiLSTM). The CNN provides the high-level features extracted at the pooling layer where the attention mechanism chooses the most relevant features and generates low-dimensional augmented features. Finally, BiLSTM is used to classify the processed sequential information. We conducted experiments on two publicly available databases to show that the proposed approach achieves the state-of-the-art results with an accuracy of 97% and 84% while being able to generate explanations. Explainability analysis has been carried out using feature visualization through PCA projection and t-SNE plots.<br></p>

show abstract

Section: Introductionmentioning

confidence: 99%

SAM: Self-augmentation mechanism for COVID-19 detection using chest X-ray Images

Usman¹,

Hoque²,

Oussalah³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Depending on the amount and diversity of sensors used during the data collection phase, several signals have been assessed and evaluated in various settings for the development of pain assessment systems. Some of the most prominently used signals constitute of the audio signal (e.g., paralinguistic vocalizations) (Tsai et al, 2016 , 2017 ; Thiam et al, 2017 ; Thiam and Schwenker, 2019 ), the video signal (e.g., facial expressions) (Rodriguez et al, 2017 ; Werner et al, 2017 ; Tavakolian and Hadid, 2019 ; Thiam et al, 2020b ), specific bio-physiological signals such as the Electrodermal Activity (EDA), the Electrocardiogram (ECG), the Electromyography (EMG), or the Respiration (RSP) signal (Walter et al, 2014 ; Campbell et al, 2019 ; Thiam et al, 2019a ), and also bodily expression signals (Dickey et al, 2002 ; Olugbade et al, 2019 ; Uddin and Canavan, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

Multi-Modal Pain Intensity Assessment Based on Physiological Signals: A Deep Learning Perspective

et al. 2021

View full text Add to dashboard Cite

Traditional pain assessment approaches ranging from self-reporting methods, to observational scales, rely on the ability of an individual to accurately assess and successfully report observed or experienced pain episodes. Automatic pain assessment tools are therefore more than desirable in cases where this specific ability is negatively affected by various psycho-physiological dispositions, as well as distinct physical traits such as in the case of professional athletes, who usually have a higher pain tolerance as regular individuals. Hence, several approaches have been proposed during the past decades for the implementation of an autonomous and effective pain assessment system. These approaches range from more conventional supervised and semi-supervised learning techniques applied on a set of carefully hand-designed feature representations, to deep neural networks applied on preprocessed signals. Some of the most prominent advantages of deep neural networks are the ability to automatically learn relevant features, as well as the inherent adaptability of trained deep neural networks to related inference tasks. Yet, some significant drawbacks such as requiring large amounts of data to train deep models and over-fitting remain. Both of these problems are especially relevant in pain intensity assessment, where labeled data is scarce and generalization is of utmost importance. In the following work we address these shortcomings by introducing several novel multi-modal deep learning approaches (characterized by specific supervised, as well as self-supervised learning techniques) for the assessment of pain intensity based on measurable bio-physiological data. While the proposed supervised deep learning approach is able to attain state-of-the-art inference performances, our self-supervised approach is able to significantly improve the data efficiency of the proposed architecture by automatically generating physiological data and simultaneously performing a fine-tuning of the architecture, which has been previously trained on a significantly smaller amount of data.

show abstract

“…This competence is termed as few-shot class-incremental learning (FSCIL) [1]. It is also crucial for deep models to acquire this ability in some real applications, such as medical image analysis [2,3,4] and autonomous driving vehicles [5,6,7]. Collecting labeled data for such applications is laborious due to several challenges, including timely and expensive processes, privacy issues, and expert knowledge demanded.…”

Section: Introductionmentioning

confidence: 99%

“…The contributions of our work are three folds: (1) For the first time, we put forward a challenging and practical learning problem called SSFSCIL, which uses unlabeled data along with labeled training data in FSCIL to improve the efficiency and performance; (2) We incorporate a simple but efficient self-training strategy in SSFSCIL for more effective semisupervised learning; (3) We perform extensive experiments on three benchmark datasets to evaluate the performance of our proposed method for image classification and introduce new baselines in this new research direction.…”

Section: Introductionmentioning

confidence: 99%

Semi-Supervised Few-Shot Class-Incremental Learning

Cui¹,

Xiong²,

Tavakolian

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

The capability of incrementally learning new classes and learning from a few examples is one of the hallmarks of human intelligence. It is crucial to endow a practical recognition system with such ability. Therefore, in this paper, we conduct pioneering work and focus on a challenging yet practical Semi-Supervised Few-Shot Class-Incremental Learning (SSFSCIL) problem, which requires CNN models incrementally learn new classes from very few labeled samples and a large number of unlabeled samples, without forgetting the previously learned ones. To address this problem, a simple and efficient solution for SSFSCIL is proposed to learn novel categories using a self-training strategy in a semi-supervised manner and avoid catastrophic forgetting by distillation-based methods. Our extensive experiments on CIFAR100, miniImageNet and CUB200 datasets demonstrate the promising performance of our proposed method, and define baselines in this new research direction.

show abstract

A Spatiotemporal Convolutional Neural Network for Automatic Pain Intensity Estimation from Facial Dynamics

Cited by 50 publications

References 36 publications

SAM: Self-augmentation mechanism for COVID-19 detection using chest X-ray Images

SAM: Self-augmentation mechanism for COVID-19 detection using chest X-ray Images

Multi-Modal Pain Intensity Assessment Based on Physiological Signals: A Deep Learning Perspective

Semi-Supervised Few-Shot Class-Incremental Learning

Contact Info

Product

Resources

About