Automatic Detection of Orofacial Impairment in Stroke

Bandini, Andrea; Green, Jordan R.; Richburg, Brian; Yunusova, Yana

doi:10.21437/interspeech.2018-2475

Cited by 22 publications

(16 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Features to identify individuals PS measured mouth range of motion and velocity, and facial symmetry (13 features). These features have been previously described and validated [12]. Features to identify individuals with ALS measured mouth range of motion and velocity, overall movement of the lower lip, mouth symmetry, and the overall roundness of lips during movement (11 features).…”

Section: Video-based Diagnosis Of Neurological Diseasesmentioning

confidence: 99%

“…Furthermore, despite significant efforts to improve FA models performance on clinical populations, there is no quantitative evidence that the improved accuracy in landmark localization leads to an improved computeraided diagnosis of neurological diseases from video based monitoring. We have shown that by using pre-trained FA models is possible to differentiate aged-matched HC from individuals PS with an accuracy of 87% [12], and age-matched HC from individuals with ALS with an accuracy close to 89% [14] using videos of speech and non-speech tasks. Based on these result, and the improved performance provided by fine-tuned FA models on clinical populations, we hypothesized that better diagnosis of neurological diseases from video based monitoring would be achieved by applying FA models fine-tuned with representative data as compared to pre-trained FA models.…”

Section: Introductionmentioning

confidence: 99%

“…In these applications, coordinates of detected facial landmarks are either used to compute a set of features or fed directly to a machine learning model (e.g. for pain detection [9]- [11], or detection of neurological diseases and motor disorders [12]- [29]) or used in pre-processing to align face images for a subsequent deep learning model (e.g. for face verification [30]).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Improving Face Alignment Accuracy on Clinical Populations and its effect on the Video-based Detection of Neurological Diseases

Guarín¹,

Bandini²,

Dempster³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Background: Automatic facial landmark localization is an essential component in many computer vision applications, including video-based detection of neurological diseases. Machine learning models for facial landmarks localization are typically trained on faces of healthy individuals, and we found that model performance is inferior when applied to faces of people with neurological diseases. Fine-tuning pre-trained models with representative images improves performance on clinical populations significantly. However, questions related to the characteristics of the database used to fine-tune the model and the clinical impact of the improved model remain. Methods: We employed the Toronto NeuroFace dataset – a dataset consisting videos of Healthy Controls (HC), individuals Post-Stroke, and individuals with Amyotrophic Lateral Sclerosis performing speech and non-speech tasks with thousands of manually annotated frames - to fine-tune a well-known deep learning-based facial landmark localization model. The pre-trained and fine-tuned models were used to extract landmark-based facial features from videos, and the facial features were used to discriminate clinical groups from HC. Results: Fine-tuning a facial landmark localization model with a diverse database that includes HC and individuals with neurological disorders resulted in significantly improved performance for all groups. Our results also showed that fine-tuning the model with representative data greatly improved the ability of the subsequent classifier to classify clinical groups vs. HC from videos. Conclusions: Using a diverse database for model fine-tuning might result in better model performance for HC and clinical groups. We demonstrated that fine-tuning a model for landmark localization with representative data results in improved detection of neurological diseases.

show abstract

Section: Video-based Diagnosis Of Neurological Diseasesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improving Face Alignment Accuracy on Clinical Populations and its effect on the Video-based Detection of Neurological Diseases

Guarín¹,

Bandini²,

Dempster³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…New methods and algorithms based on recent advancements in machine learning (ML) and computer vision (CV) have been developed to assess orofacial function objectively and extract clinically useful information automatically from videos and photographs of patients [3][4][5][6][7][8][9][10]. These techniques track orofacial movements without using facial markers or specialized equipment.…”

mentioning

confidence: 99%

“…Different from laboratory tests, these techniques are inexpensive, easy-to-use, readily available, and can be used at-home. Our team have used these methods to evaluate the effectiveness of surgical treatments for Bell's Palsy [4,5,8], and to extract biomarkers for detection and assessment of orofacial deficits in Parkinson's disease [10,11], amyotrophic lateral sclerosis [6], and stroke [7]. These techniques have the potential to revolutionize the assessment of orofacial deficits as they would allow frequent, cost-effective, objective in-home monitoring of disease progression and treatment effectiveness across various neurological diseases affecting the orofacial musculature.…”

mentioning

confidence: 99%

Automatic Facial Landmark Localization in Clinical Populations - Improving Model Performance with a Small Dataset

Guarín

Taati

Hadlock

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Background Automatic facial landmark localization in videos is an important first step in many computer vision applications, including the objective assessment of orofacial function. Convolutional neural networks (CNN) for facial landmarks localization are typically trained on faces of healthy and young adults, so model performance is inferior when applied to faces of older adults or people with diseases that affect facial movements, a phenomenon known as algorithmic bias. Fine-tuning pre-trained CNN models with representative data is a well-known technique used to reduce algorithmic bias and improve performance on clinical populations. However, the question of how much data is needed to properly fine-tune the model remains. Methods In this paper, we fine-tuned a popular CNN model for automatic facial landmarks localization using different number of manually annotated photographs from patients with facial palsy and evaluated the effects of the number of photographs used for model fine-tuning in the model performance by computing the normalized root mean squared error between the facial landmarks positions predicted by the model and those provided by manual annotators. Furthermore, we studied the effect of annotator bias by fine-tuning and evaluating the model with data provided by multiple annotators. Results Our results showed that fine-tuning the model with as little as 8 photographs from a single patient significantly improved the model performance on other individuals from the same clinical population, and that the best performance was achieved by fine-tuning the model with 320 photographs from 40 patients. Using more photographs for fine-tuning did not improve the model performance further. Regarding the annotator bias, we found that fine-tuning a CNN model with data from one annotator resulted in models biased against other annotators; our results also showed that this effect can be diminished by averaging data from multiple annotators. Conclusions It is possible to remove the algorithmic bias of a\textbf{depth} CNN model for automatic facial landmark localization using data from only 40 participants (total of 320 photographs). These results pave the way to future clinical applications of CNN models for the automatic assessment of orofacial function in different clinical populations, including patients with Parkinson’s disease and stroke.

show abstract