XraySyn: Realistic View Synthesis From a Single Radiograph Through CT Priors

Abstract

A radiograph visualizes the internal anatomy of a patient through the use of X-ray, which projects 3D information onto a 2D plane. Hence, radiograph analysis naturally requires physicians to relate their prior knowledge about 3D human anatomy to 2D radiographs. Synthesizing novel radiographic views in a small range can assist physicians in interpreting anatomy more reliably; however, radiograph view synthesis is heavily ill-posed, lacking in paired data, and lacking in differentiable operations to leverage learning-based approaches. To address these problems, we use Computed Tomography (CT) for radiograph simulation and design a differentiable projection algorithm, which enables us to achieve geometrically consistent transformations between the radiography and CT domains. Our method, XraySyn, can synthesize novel views on real radiographs through a combination of realistic simulation and finetuning on real radiographs. To the best of our knowledge, this is the first work on radiograph view synthesis. We show that by gaining an understanding of radiography in 3D space, our method can be applied to radiograph bone extraction and suppression without requiring groundtruth bone labels.

X-ray dissectography enables stereotography

“…As x-ray radiography has the lowest radiation dose, the fastest imaging speed, and the lowest price, researchers have been focusing on improving the radiogram quality. Currently, there are mainly two ways for this purpose: 1) suppressing interfered structures [1][2][3][4] or enhancing related structures, 1,5 and 2) generating 3D volumes. [6][7][8] It is well known that superimposed anatomical organs in 2D radiographs significantly complicate signal detection, such as for diagnosis of lung diseases.…”

“…In early studies 9, 10 model-based methods were developed to suppress ribs in chest radiographs, some of which require manually annotated bone masks. In recent years, deep learning methods [1][2][3][4] were proposed for suppression of ribs by leveraging 3D CT prior. Instead of suppressing the ribs in CXR images, Gozes and Greenspan proposed to enhance lung structures by extracting the extracted lungs first and then adding the result back with a scaling factor.…”

“…Among the above-surveyed methods, those methods that are for suppressing/enhancing specific structures are mainly intended to improve the performance of classification [1][2][3][4] and detection 5 from a single radiograph without providing 3D information. On the other hand, although the rest of the existing methods that map 2D radiograms to 3D CT volumes achieved remarkable results, they cannot reconstruct structures accurately and reliably, since their clinical utilities have not been demonstrated so far.…”