Analysis of 3D Facial Dysmorphology in Genetic Syndromes from Unconstrained 2D Photographs

Tu, Liyun; Porras, Antonio R.; Boyle, Alec; Linguraru, Marius George

doi:10.1007/978-3-030-00928-1_40

Cited by 11 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Both these works used 3D data. However, the capability of 3D reconstruction from 2D images was explored for the screening of acromegaly [Learned-Miller et al 2006] and genetic disorders [Tu et al 2018].…”

Section: Medical Applicationsmentioning

confidence: 99%

3D Morphable Face Models—Past, Present, and Future

et al. 2020

View full text Add to dashboard Cite

In this article, we provide a detailed survey of 3D Morphable Face Models over the 20 years since they were first proposed. The challenges in building and applying these models, namely, capture, modeling, image formation, and image analysis, are still active research topics, and we review the state-of-the-art in each of these areas. We also look ahead, identifying unsolved challenges, proposing directions for future research, and highlighting the broad range of current and future applications.

show abstract

Section: Medical Applicationsmentioning

confidence: 99%

3D Morphable Face Models—Past, Present, and Future

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Three-dimensional (3D) face reconstruction from uncalibrated two-dimensional (2D) images is a long-standing problem in computer vision with many different applications, such as face recognition [1,2,3,4], face alignment [5,6,7,8], image edition [9,10,11], face animation [12,13,14], age estimation [15], or medical diagnostic purposes [16,17,18]. Despite its great amount of advantages, 3D face reconstruction is a very challenging task since it is inherently ill-posed.…”

Section: Introductionmentioning

confidence: 99%

“…Although many different approaches have been proposed to tackle the 3D face reconstruction problem, none of them provides a solution for 3D face reconstruction of babies. This application is especially relevant to enable medical diagnosis based on cranio-facial imaging data [16,24,25]. A challenge is that the facial geometry of babies is very different from that older children or adults.…”

Section: Introductionmentioning

confidence: 99%

BabyNet: Reconstructing 3D faces of babies from uncalibrated photographs

Morales¹,

Porras²,

Linguraru³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We present a 3D face reconstruction system that aims at recovering the 3D facial geometry of babies from uncalibrated photographs, BabyNet. Since the 3D facial geometry of babies differs substantially from that of adults, baby-specific facial reconstruction systems are needed. BabyNet consists of two stages: 1) a 3D graph convolutional autoencoder learns a latent space of the baby 3D facial shape; and 2) a 2D encoder that maps photographs to the 3D latent space based on representative features extracted using transfer learning. In this way, using the pre-trained 3D decoder, we can recover a 3D face from 2D images. We evaluate BabyNet and show that 1) methods based on adult datasets cannot model the 3D facial geometry of babies, which proves the need for a babyspecific method, and 2) BabyNet outperforms classical model-fitting methods even when a baby-specific 3D morphable model, such as BabyFM, is used.

show abstract

“…Traditionally performed by a physician, the advents of computer vision and machine learning in medicine enable rapid and automated assessment of a patient's facial traits [3,4]. Numerous facial phenotyping systems have been developed with the potential to aid the diagnostic processes in medical genetics [5][6][7][8][9][10][11][12]. DeepGestalt, the neural network behind Face2Gene CLINIC, which was trained on more than 17,106 images, is thus far the best-investigated and most convenient to use application [11].…”

Section: Introductionmentioning

confidence: 99%

Efficiency of Computer-Aided Facial Phenotyping (DeepGestalt) in Individuals With and Without a Genetic Syndrome: Diagnostic Accuracy Study (Preprint)

Pantel¹,

Hajjir²,

Danyel³

et al. 2020

Preprint

View full text Add to dashboard Cite

BACKGROUND Collectively, an estimated 5% of the population have a genetic disease. Many of them feature characteristics that can be detected by facial phenotyping. Face2Gene CLINIC is an online app for facial phenotyping of patients with genetic syndromes. DeepGestalt, the neural network driving Face2Gene, automatically prioritizes syndrome suggestions based on ordinary patient photographs, potentially improving the diagnostic process. Hitherto, studies on DeepGestalt’s quality highlighted its sensitivity in syndromic patients. However, determining the accuracy of a diagnostic methodology also requires testing of negative controls. OBJECTIVE The aim of this study was to evaluate DeepGestalt's accuracy with photos of individuals with and without a genetic syndrome. Moreover, we aimed to propose a machine learning–based framework for the automated differentiation of DeepGestalt’s output on such images. METHODS Frontal facial images of individuals with a diagnosis of a genetic syndrome (established clinically or molecularly) from a convenience sample were reanalyzed. Each photo was matched by age, sex, and ethnicity to a picture featuring an individual without a genetic syndrome. Absence of a facial gestalt suggestive of a genetic syndrome was determined by physicians working in medical genetics. Photos were selected from online reports or were taken by us for the purpose of this study. Facial phenotype was analyzed by DeepGestalt version 19.1.7, accessed via Face2Gene CLINIC. Furthermore, we designed linear support vector machines (SVMs) using Python 3.7 to automatically differentiate between the 2 classes of photographs based on DeepGestalt's result lists. RESULTS We included photos of 323 patients diagnosed with 17 different genetic syndromes and matched those with an equal number of facial images without a genetic syndrome, analyzing a total of 646 pictures. We confirm DeepGestalt’s high sensitivity (top 10 sensitivity: 295/323, 91%). DeepGestalt’s syndrome suggestions in individuals without a craniofacially dysmorphic syndrome followed a nonrandom distribution. A total of 17 syndromes appeared in the top 30 suggestions of more than 50% of nondysmorphic images. DeepGestalt’s top scores differed between the syndromic and control images (area under the receiver operating characteristic [AUROC] curve 0.72, 95% CI 0.68-0.76; <i>P</i><.001). A linear SVM running on DeepGestalt’s result vectors showed stronger differences (AUROC 0.89, 95% CI 0.87-0.92; <i>P</i><.001). CONCLUSIONS DeepGestalt fairly separates images of individuals with and without a genetic syndrome. This separation can be significantly improved by SVMs running on top of DeepGestalt, thus supporting the diagnostic process of patients with a genetic syndrome. Our findings facilitate the critical interpretation of DeepGestalt’s results and may help enhance it and similar computer-aided facial phenotyping tools.

show abstract

Analysis of 3D Facial Dysmorphology in Genetic Syndromes from Unconstrained 2D Photographs

Cited by 11 publications

References 11 publications

3D Morphable Face Models—Past, Present, and Future

3D Morphable Face Models—Past, Present, and Future

BabyNet: Reconstructing 3D faces of babies from uncalibrated photographs

Efficiency of Computer-Aided Facial Phenotyping (DeepGestalt) in Individuals With and Without a Genetic Syndrome: Diagnostic Accuracy Study (Preprint)

Contact Info

Product

Resources

About