Generating Synthetic X-Ray Images of a Person from the Surface Geometry

Abstract

We present a novel framework that learns to predict human anatomy from body surface. Specifically, our approach generates a synthetic X-ray image of a person only from the person's surface geometry. Furthermore, the synthetic Xray image is parametrized and can be manipulated by adjusting a set of body markers which are also generated during the X-ray image prediction. With the proposed framework, multiple synthetic X-ray images can easily be generated by varying surface geometry. By perturbing the parameters, several additional synthetic X-ray images can be generated from the same surface geometry. As a result, our approach offers a potential to overcome the training data barrier in the medical domain. This capability is achieved by learning a pair of networks -one learns to generate the full image from the partial image and a set of parameters, and the other learns to estimate the parameters given the full image. During training, the two networks are trained iteratively such that they would converge to a solution where the predicted parameters and the full image are consistent with each other. In addition to medical data enrichment, our framework can also be used for image completion as well as anomaly detection. 1

Synthesis of COVID-19 chest X-rays using unpaired image-to-image translation

“…Image synthesis methodologies have also been proposed in the context of chest X-rays (Teixeira et al 2018). Our work is significantly different in the sense that we are specifically interested in synthesizing a particular class, whereas in Teixeira et al (2018) X-rays are generated from surface geometry for landmark detection tasks.…”

“…Image synthesis methodologies have also been proposed in the context of chest X-rays (Teixeira et al 2018). Our work is significantly different in the sense that we are specifically interested in synthesizing a particular class, whereas in Teixeira et al (2018) X-rays are generated from surface geometry for landmark detection tasks. While some generative methods only require paired data in the source domain with target domain consisting of unlabeled examples, Cohen et al (2018) have demonstrated that the phenomenon of hallucinating features (e.g., adding or removing tumors leading to a semantic change) leads to a high bias in these domain adaptation techniques.…”

Autonomous Robotic Point-of-Care Ultrasound Imaging for Monitoring of COVID-19–Induced Pulmonary Diseases

“…Recent research in deep learning has shown encouraging progress in detecting 3D landmarks from surface data ( Papazov et al, 2015 ; He et al, 2019 ; Liu et al, 2019 ). Teixeira et al (2018 ) introduced a method to estimate 2D positions of internal body landmarks from a 2.5D surface image of the torso generated by orthographically projecting the 3D skin surface. Such methods, however, do not generalize to 3D data, since it would require representing landmark likelihood as a Gaussian distribution over a dense 3D lattice, and the memory required for representing 60 such lattices, one for each landmark, would be overwhelming.…”

The Design of X-Ray Film Reader with Film Presence Detector

“…Reduction of X-Ray Dose in Chest Tomosynthesis was investigated by Miroshnychenko [18]. Generating a Synthetic X-Ray Image of a Person from Surface Geometry was investigated by Teixeira [19]. Classification of modalities and detection of concepts on medical images using distance transfer learning was investigated by Singh [20].…”