Enabling machine learning in X-ray-based procedures via realistic simulation of image formation

Unberath, Mathias; Zaech, Jan-Nico; Gao, Cong; Bier, Bastian; Goldmann, Florian; Lee, Sing Chun; Fotouhi, Javad; Taylor, Russell H.; Armand, Mehran; Navab, Nassir

doi:10.1007/s11548-019-02011-2

Cited by 50 publications

(49 citation statements)

References 40 publications

(52 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For clinical and patient-specific applications, existing CT scansif availablecould be segmented and labeled automatically using a deep learning approach. [45][46][47] As an alternative, Hounsfield units could be mapped to precalibrated densities and tissue labels. Besides, a comparable patient model from a database, such as the XCAT family, could be considered.…”

Section: Discussionmentioning

confidence: 99%

Physics‐driven learning of x‐ray skin dose distribution in interventional procedures

et al. 2019

View full text Add to dashboard Cite

Purpose Radiation doses accumulated during very complicated image‐guided x‐ray procedures have the potential to cause stochastic, but also deterministic effects, such as skin rashes or even hair loss. To monitor and reduce radiation‐related risks to patients’ skin, x‐ray imaging devices are equipped with online air kerma monitoring components. Traditionally, such measurements have been used to estimate skin entrance dose by (a) estimating air kerma at the interventional reference point (IRP), (b) forward projecting the dose distribution, and (c) considering a backscatter factor among other correction factors. Unfortunately, the complicated interaction between incident x‐ray photons, secondary electrons, and skin tissue cannot be properly accounted for by assuming a linear relationship between forward projected air kerma and a backscatter factor. Gold standard skin dose models are therefore determined using Monte Carlo (MC) techniques. However, MC simulations are computationally complex in general and possible acceleration mainly depends on the employed hardware and variance reduction techniques. To obtain reliable and fast dose estimates, we propose to combine MC‐based simulations with learning‐based methods. Methods The basic idea of our method is to approximate the radiation physics to calculate a first‐order exposure estimate quickly. This initial estimate is then refined using prior knowledge derived from MC simulations. To this end, the primary photon propagation inside a voxelized patient model is estimated using a less accurate but fast photon ray casting (RC) simulation based on the Beer–Lambert law. The results of the RC simulation are then fed into a convolutional neural network (CNN), which maps the propagation of primary photons to the dose deposition inside the patient model. Additionally, the patient model itself including anatomy and material properties, such as mass density and mass energy‐absorption coefficients, are fed into the CNN as well. The CNN is trained using smoothed results of MC simulations as output and RC simulations of identical imaging settings and patient models as input. Results In total, 163 MC and associated RC simulations are carried out for the head, thorax, abdomen, and pelvis in three different voxel phantoms. We used 108 or 109 primarily emitted photons sampled from a 125 kV peak voltage spectrum, respectively. Edge‐preserving smoothing (EPS) is applied to reduce (a) general stochastic uncertainties and (b) stochastic uncertainty concerning MC simulations of less primary photons. The CNN is trained using seven imaging settings of the abdomen in a single phantom. Testing its performance on the remaining datasets, the CNN is capable of estimating skin dose with an error of below 10% for the majority of test cases. Conclusion The combination of deep neural networks and MC simulation of particle physics has the potential to decrease the computational complexity of accurate skin dose estimation. The proposed approach can provide dose distributions in under one second when runni...

show abstract

Section: Discussionmentioning

confidence: 99%

Physics‐driven learning of x‐ray skin dose distribution in interventional procedures

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Training: Training is performed on realistic digitally reconstructed radiographs (DRRs) generated from CT using the open-source tool DeepDRR [11]. The pipeline was chosen as it enables the simulation of metal artifact as well as the transfer to real data with only low degradation of prediction performance [10]. For DRR generation, five chest CT-volumes were obtained from the Cancer Imaging Archive.…”

Section: Methodsmentioning

confidence: 99%

Learning to Avoid Poor Images: Towards Task-aware C-arm Cone-beam CT Trajectories

Zaech

Gao

Bier

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Metal artifacts in computed tomography (CT) arise from a mismatch between physics of image formation and idealized assumptions during tomographic reconstruction. These artifacts are particularly strong around metal implants, inhibiting widespread adoption of 3D cone-beam CT (CBCT) despite clear opportunity for intra-operative verification of implant positioning, e. g. in spinal fusion surgery. On synthetic and real data, we demonstrate that much of the artifact can be avoided by acquiring better data for reconstruction in a task-aware and patientspecific manner, and describe the first step towards the envisioned taskaware CBCT protocol. The traditional short-scan CBCT trajectory is planar, with little room for scene-specific adjustment. We extend this trajectory by autonomously adjusting out-of-plane angulation. This enables C-arm source trajectories that are scene-specific in that they avoid acquiring "poor images", characterized by beam hardening, photon starvation, and noise. The recommendation of ideal out-of-plane angulation is performed on-the-fly using a deep convolutional neural network that regresses a detectability-rank derived from imaging physics.

show abstract

“…While large scale acquisition of highly structured data is tractable for some interventional applications, particularly ultrasound [61], [62], most other approaches rely on synthetic data generation from physical models of the scene. This paradigm is attractive because all quantities of interest are precisely known by design, however, if simulation is performed naïvely, AI models trained on synthetic data will not generalize to clinically acquired images because of the large domain mismatch paired with poor generalizability of today's models [57]. Three complementary ways have recently been shown to mitigate this problem.…”

Section: B Simulation-based Trainingmentioning

confidence: 99%

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions

et al. 2020

Self Cite

View full text Add to dashboard Cite

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theatres are extremely complex and typically rely on poorly integrated intra-operative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer assisted interventions, we highlight the crucial need to take context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer assisted intervention, or CAI4CAI, arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human-AI actor team; how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision making ultimately producing more precise and reliable interventions.Index Terms-Artificial intelligence, computer assisted interventions, interventional workflow, intra-operative imaging, surgical planning, data fusion, surgical scene understanding, contextaware user interface, machine and deep learning, surgical data science

show abstract

Enabling machine learning in X-ray-based procedures via realistic simulation of image formation

Cited by 50 publications

References 40 publications

Physics‐driven learning of x‐ray skin dose distribution in interventional procedures

Physics‐driven learning of x‐ray skin dose distribution in interventional procedures

Learning to Avoid Poor Images: Towards Task-aware C-arm Cone-beam CT Trajectories

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions

Contact Info

Product

Resources

About