Detecting hypomimia symptoms by selfie photo analysis

Grammatikopoulou, Athina; Grammalidis, Nikos; Bostantjopoulou, Sevasti; Katsarou, Zoe

doi:10.1145/3316782.3322756

Cited by 24 publications

(13 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bandini et al analyzed facial expressions in PD through video-based automatic methods ( 22 ). Grammatikoupoulou et al developed PD detection tests, based on the interaction of users with everyday technological devices to quantify the progressive decrease of variability of facial expressions in early PD patients by analyzing patterns emerging from selfie-photos ( 23 ). Rajnoha et al proposed algorithms to detect hypomimia based on face recognition methods ( 24 ).…”

Section: Current State Of Digital Biomarkers In Parkinson's Diseasementioning

confidence: 99%

Leveraging the Potential of Digital Technology for Better Individualized Treatment of Parkinson's Disease

et al. 2022

View full text Add to dashboard Cite

Recent years have witnessed a strongly increasing interest in digital technology within medicine (sensor devices, specific smartphone apps) and specifically also neurology. Quantitative measures derived from digital technology could provide Digital Biomarkers (DMs) enabling a quantitative and continuous monitoring of disease symptoms, also outside clinics. This includes the possibility to continuously and sensitively monitor the response to treatment, hence opening the opportunity to adapt medication pathways quickly. In addition, DMs may in the future allow early diagnosis, stratification of patient subgroups and prediction of clinical outcomes. Thus, DMs could complement or in certain cases even replace classical examiner-based outcome measures and molecular biomarkers measured in cerebral spinal fluid, blood, urine, saliva, or other body liquids. Altogether, DMs could play a prominent role in the emerging field of precision medicine. However, realizing this vision requires dedicated research. First, advanced data analytical methods need to be developed and applied, which extract candidate DMs from raw signals. Second, these candidate DMs need to be validated by (a) showing their correlation to established clinical outcome measures, and (b) demonstrating their diagnostic and/or prognostic value compared to established biomarkers. These points again require the use of advanced data analytical methods, including machine learning. In addition, the arising ethical, legal and social questions associated with the collection and processing of sensitive patient data and the use of machine learning methods to analyze these data for better individualized treatment of the disease, must be considered thoroughly. Using Parkinson's Disease (PD) as a prime example of a complex multifactorial disorder, the purpose of this article is to critically review the current state of research regarding the use of DMs, discuss open challenges and highlight emerging new directions.

show abstract

Section: Current State Of Digital Biomarkers In Parkinson's Diseasementioning

confidence: 99%

Leveraging the Potential of Digital Technology for Better Individualized Treatment of Parkinson's Disease

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Analyses were carried out three times for each set of features. As shown in Table 4, when only using GFs, Parkinson's hypomimia was primarily identified using variables based on the human eyes and mouth, such as D (19,21), D (13,15), and θ(47,46,45). When using TFs, areas of hypomimia-interest were located at the chin and the eyes.…”

Section: Importance Analysismentioning

confidence: 99%

“…Professional indicators, such as the Ekman score (5) and the Euclidean distance (12), can be calculated to evaluate emotional expressivity using the foregoing statistical features. In addition, extracted features can be directly fed into machine learning models (12,(19)(20)(21) or supervised learning models (22,23), to distinguish patients with the hypomimia symptom from healthy people. However, to the best of our knowledge, the clinical diagnosis of hypomimia is still dependent on inhospital clinical observations and several criteria (24)(25)(26)(27).…”

Section: Introductionmentioning

confidence: 99%

Detection of hypomimia in patients with Parkinson’s disease via smile videos

Ge¹,

Lin²,

Yin³

et al. 2021

Ann Transl Med

View full text Add to dashboard Cite

Background: Parkinson's disease (PD) is a neurodegenerative disease characterized by the impairment of facial expression, known as hypomimia. Hypomimia has serious impacts on patients' ability to communicate, and it is difficult to detect at early stages of the disease. Furthermore, due to bradykinesia or other reasons, it is inconvenient for PD patients to visit the hospital. Therefore, it is appealing to develop an auxiliary diagnostic method that remotely detects hypomimia. Methods:We proposed an automatic detection system for Parkinson's hypomimia based on facial expressions (DSPH-FE). DSPH-FE provides a convenient remote service for those who potentially suffer from hypomimia and only requires patients to input their facial videos. Specifically, patients can detect hypomimia through two aspects: geometric features and texture features. Geometric features focus on visually representing structures of facial muscles. Facial expression factors (FEFs) are used as the first metric to quantify the current activation state of the facial muscles. Facial expression change factors (FECFs) are subsequently used as the second metric to calculate the moving trajectories of the activation states in the videos. Geometric features primarily concentrate on spatial information, with little involvement of temporal information. Thus, the extended histogram of oriented gradients (HOG) algorithm is introduced. This algorithm can extract texture features within multiple continuous frames and incorporate the temporal information into the features. Finally, these features are applied to four machine learning algorithms to model the relationship between these features and hypomimia.Results: The DSPH-FE detection system achieved the best performance when concatenating geometric features and texture features, resulting in a F1 score of 0.9997. The best F1 scores achieved with geometric features and texture features were 0.8286 and 0.9446, respectively. This indicated that both geometric features and texture features have an ability to predict hypomimia, and demonstrated that temporal information can boost the model performance. Thus, DSPH-FE is an effective supportive tool in the medical management of PD patients. Conclusions:Comprehensive experiments demonstrated that proposed features fit well with real-world videos and are beneficial in the clinical diagnosis of hypomimia. In particular, hypomimia had a greater impact on eyes and mouths when patients are smiling.

show abstract

“…They estimated a quantitative measure from the neutral expression in a subset of basic emotions (happiness, anger, disgust, and sadness), considering both the actuated and the imitated ones. Grammatikopoulou and colleagues [ 21 ] proposed an innovative evaluation of this symptom in PD based on images captured by smartphones. Two different indexes of hypomimia were developed without discriminating among different emotions.…”

Section: Introductionmentioning

confidence: 99%

Quantitative Evaluation of Hypomimia in Parkinson’s Disease: A Face Tracking Approach

Pegolo

Volpe²,

Cucca

et al. 2022

Sensors

View full text Add to dashboard Cite

Parkinson’s disease (PD) is a neurological disorder that mainly affects the motor system. Among other symptoms, hypomimia is considered one of the clinical hallmarks of the disease. Despite its great impact on patients’ quality of life, it remains still under-investigated. The aim of this work is to provide a quantitative index for hypomimia that can distinguish pathological and healthy subjects and that can be used in the classification of emotions. A face tracking algorithm was implemented based on the Facial Action Coding System. A new easy-to-interpret metric (face mobility index, FMI) was defined considering distances between pairs of geometric features and a classification based on this metric was proposed. Comparison was also provided between healthy controls and PD patients. Results of the study suggest that this index can quantify the degree of impairment in PD and can be used in the classification of emotions. Statistically significant differences were observed for all emotions when distances were taken into account, and for happiness and anger when FMI was considered. The best classification results were obtained with Random Forest and kNN according to the AUC metric.

show abstract

Detecting hypomimia symptoms by selfie photo analysis

Cited by 24 publications

References 11 publications

Leveraging the Potential of Digital Technology for Better Individualized Treatment of Parkinson's Disease

Leveraging the Potential of Digital Technology for Better Individualized Treatment of Parkinson's Disease

Detection of hypomimia in patients with Parkinson’s disease via smile videos

Quantitative Evaluation of Hypomimia in Parkinson’s Disease: A Face Tracking Approach

Contact Info

Product

Resources

About