Kinect-driven Patient-specific Head, Skull, and Muscle Network Modelling for Facial Palsy Patients

Nguyen, Tan-Nhu; Dakpe, Stéphanie; Tho, M.C. Ho Ba; Dao, Tien Tuan

doi:10.1016/j.cmpb.2020.105846

Cited by 11 publications

(19 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The subject performed several trials with neutral positions and facial mimic positions, such as smile, [e], and [u] pronunciation. Our developed Kinect-based computer vision system [ 5 ] was used to capture the high density (HD) point clouds of the face as well as the RGB image from Kinect sensors. The images were captured where each subject was positioned in front of the camera with a distance of about 1 m. The RGB image was used for 3D shape reconstruction with the deep learning models.…”

Section: Methodsmentioning

confidence: 99%

“…This leads to unwanted facial movements, such as dysfunctionalities of speaking, eating, and the unnatural relaxation of mouth corners drop, eyelid closure, and asymmetrical facial expressions [ 3 , 4 ]. Recently, computer-aided decision systems have been developed to provide objective and quantitative indicators to better diagnose and to optimize the rehabilitation program [ 5 ]. The 3D reconstruction of an accurate face model is essential for providing reliable feedback.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients

Nguyen

Dakpe

et al. 2022

Bioengineering

Self Cite

View full text Add to dashboard Cite

The 3D reconstruction of an accurate face model is essential for delivering reliable feedback for clinical decision support. Medical imaging and specific depth sensors are accurate but not suitable for an easy-to-use and portable tool. The recent development of deep learning (DL) models opens new challenges for 3D shape reconstruction from a single image. However, the 3D face shape reconstruction of facial palsy patients is still a challenge, and this has not been investigated. The contribution of the present study is to apply these state-of-the-art methods to reconstruct the 3D face shape models of facial palsy patients in natural and mimic postures from one single image. Three different methods (3D Basel Morphable model and two 3D Deep Pre-trained models) were applied to the dataset of two healthy subjects and two facial palsy patients. The reconstructed outcomes were compared to the 3D shapes reconstructed using Kinect-driven and MRI-based information. As a result, the best mean error of the reconstructed face according to the Kinect-driven reconstructed shape is 1.5 ± 1.1 mm. The best error range is 1.9 ± 1.4 mm when compared to the MRI-based shapes. Before using the procedure to reconstruct the 3D faces of patients with facial palsy or other facial disorders, several ideas for increasing the accuracy of the reconstruction can be discussed based on the results. This present study opens new avenues for the fast reconstruction of the 3D face shapes of facial palsy patients from a single image. As perspectives, the best DL method will be implemented into our computer-aided decision support system for facial disorders.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients

Nguyen

Dakpe

et al. 2022

Bioengineering

Self Cite

View full text Add to dashboard Cite

show abstract

“…Plastic and Reconstructive Surgery • September 2022 three-dimensional data without multisensor setups, and four-dimensional files are not yet optimized, hindering storage and data transfer. 8,10 OpenFAS anthropometric calculations for each facial movement are not yet confirmed or validated and have not been outlined in this study. Most notably, we found that only 61.8 percent of landmarks were placed within what we deemed an acceptable accuracy threshold.…”

Section: Ementioning

confidence: 95%

“Introduction of a low-cost and automated four-dimensional assessment system of the face.”

Petrides

Joy

Dolk

et al. 2022

Plastic &Amp; Reconstructive Surgery

View full text Add to dashboard Cite

Summary: Existing automated objective grading systems either fail to consider the face’s complex three-dimensional morphology or have poor feasibility and usability. Consumer-based red, green, and blue depth sensors or smartphone integrated three-dimensional hardware can inexpensively collect detailed four-dimensional facial data in real time but are yet to be incorporated into a practical system. This study aimed to evaluate the feasibility of a proof-of-concept automated four-dimensional facial assessment system using a red/green/blue depth sensor (OpenFAS) for use in a standard clinical environment. This study was performed on healthy adult volunteers and patients with facial nerve palsy. The setup consists of the Intel (Santa Clara, Calif.) RealSense SR300 connected to a laptop running the OpenFAS application. The subject sequentially mimics the facial expressions shown on screen. Each frame is landmarked and automatic anthropometric calculations are performed. Any errors during each session were noted. Landmarking accuracy was estimated by comparing the ground-truth position of landmarks annotated manually with those placed automatically. Eighteen participants were included in the study, nine healthy participants and nine patients with facial nerve palsy. Each session was standardized at approximately 106 seconds. A total of 61.8 percent of landmarks were automatically annotated within approximately 1.575 mm of their ground-truth locations. The findings support that OpenFAS is usable and feasible in routine settings, laying down the critical groundwork for a facial assessment system that addresses the shortcomings of existing tools. The iteration of OpenFAS presented in this study is nascent; future work, including improvements to landmarking accuracy, analyses components, and red/green/blue depth technology, is required before clinical application.

show abstract

“…We also expect our method's cost-effectiveness to increase as the technology develops. Markerless systems might emerge, as has been the case for gait analysis (Nguyen et al, 2021). However, it should be borne in mind that markerless gait monitoring systems are mainly being developed for home-based self-monitoring and rehabilitation.…”

Section: Study Limitationsmentioning

confidence: 99%

Quantified analysis of facial movement: A reference for clinical applications

et al. 2023

View full text Add to dashboard Cite

Most techniques for evaluating unilateral impairments in facial movement yield subjective measurements. The objective of the present study was to define a reference dataset and develop a visualization tool for clinical assessments. In this prospective study, a motion capture system was used to quantify facial movements in 30 healthy adults and 2 patients. We analyzed the displacements of 105 reflective markers placed on the participant's face during five movements (M1–M5). For each marker, the primary endpoint was the maximum amplitude of displacement from the static position (M0) in an analysis of variance. The measurement precision was 0.1 mm. Significant displacements of markers were identified for M1–M5, and displacement patterns were defined. The patients and age‐matched healthy participants were compared with regard to the amplitude of displacement. We created a new type of radar plot to visually represent the diagnosis and facilitate effective communication between medical professionals. In proof‐of‐concept experiments, we collected quantitative data on patients with facial palsy and created a patient‐specific radar plot. Our new protocol for clinical facial motion capture (“quantified analysis of facial movement,” QAFM) was accurate and should thus facilitate the long‐term clinical follow‐up of patients with facial palsy. To take account of the limitations affecting the comparison with the healthy side, we created a dataset of healthy facial movements; our method might therefore be applicable to other conditions in which movements on one or both sides of the face are impaired. The patient‐specific radar plot enables clinicians to read and understand the results rapidly.

show abstract

Kinect-driven Patient-specific Head, Skull, and Muscle Network Modelling for Facial Palsy Patients

Cited by 11 publications

References 66 publications

Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients

Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients

“Introduction of a low-cost and automated four-dimensional assessment system of the face.”

Quantified analysis of facial movement: A reference for clinical applications

Contact Info

Product

Resources

About