Uncertainty quantification in medical image synthesis

Jin, Bangti; Barbano, Riccardo; Arridge, Simon; Tanno, Ryutaro

doi:10.1016/b978-0-12-824349-7.00033-5

Cited by 6 publications

(11 citation statements)

References 121 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such decomposition sheds light on the possible sources of synthesis error 41 and offers valuable guidance on training data selection and algorithm design. Bayesian approaches provide a theoretical framework for uncertainty estimation, but are often impractical and computationally intensive 41,42 . Numerous efforts have been devoted to developing approximation techniques such as Laplace approximation, 43 variational inference, 44 and Monte Carlo (MC) dropout 45 .…”

Section: Introductionmentioning

confidence: 99%

“…Predictive uncertainty can be broadly categorized into two types: (1) aleatoric uncertainty, which refers to ambiguity in prediction caused by intrinsic noise in the input and (2) epistemic uncertainty, which is incurred by a lack of knowledge in the training data (e.g., previously unseen pathologies) or inadequacy in the DL model. Such decomposition sheds light on the possible sources of synthesis error 41 and offers valuable guidance on training data selection and algorithm design. Bayesian approaches provide a theoretical framework for uncertainty estimation, but are often impractical and computationally intensive 41,42 .…”

Section: Introductionmentioning

confidence: 99%

“…Improved prediction performance and reliability via explicit modeling of uncertainty have also been demonstrated in the context of depth regression, 46 deformable registration, 47 and image segmentation 48 . The full utility of uncertainty information has yet to be widely investigated in medical image synthesis and reconstruction 41 …”

Section: Introductionmentioning

confidence: 99%

“…48 The full utility of uncertainty information has yet to be widely investigated in medical image synthesis and reconstruction. 41 In this work, we present a DL reconstruction framework (referred to as DL-Recon) to improve intraoperative CBCT. The major contribution of this framework is the use of DL uncertainty as a voxel-wise weighting map to combine physics-based models with DL image synthesis.…”

mentioning

confidence: 99%

“…Bayesian approaches provide a theoretical framework for uncertainty estimation, but are often impractical and computationally intensive. 41,42 Numerous efforts have been devoted to developing approximation techniques such as Laplace approximation, 43 variational inference, 44 and Monte Carlo (MC) dropout. 45 Improved prediction performance and reliability via explicit modeling of uncertainty have also been demonstrated in the context of depth regression, 46 deformable registration, 47 and image segmentation.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Combining physics‐based models with deep learning image synthesis and uncertainty in intraoperative cone‐beam CT of the brain

et al. 2023

View full text Add to dashboard Cite

Background Image‐guided neurosurgery requires high localization and registration accuracy to enable effective treatment and avoid complications. However, accurate neuronavigation based on preoperative magnetic resonance (MR) or computed tomography (CT) images is challenged by brain deformation occurring during the surgical intervention. Purpose To facilitate intraoperative visualization of brain tissues and deformable registration with preoperative images, a 3D deep learning (DL) reconstruction framework (termed DL‐Recon) was proposed for improved intraoperative cone‐beam CT (CBCT) image quality. Methods The DL‐Recon framework combines physics‐based models with deep learning CT synthesis and leverages uncertainty information to promote robustness to unseen features. A 3D generative adversarial network (GAN) with a conditional loss function modulated by aleatoric uncertainty was developed for CBCT‐to‐CT synthesis. Epistemic uncertainty of the synthesis model was estimated via Monte Carlo (MC) dropout. Using spatially varying weights derived from epistemic uncertainty, the DL‐Recon image combines the synthetic CT with an artifact‐corrected filtered back‐projection (FBP) reconstruction. In regions of high epistemic uncertainty, DL‐Recon includes greater contribution from the FBP image. Twenty paired real CT and simulated CBCT images of the head were used for network training and validation, and experiments evaluated the performance of DL‐Recon on CBCT images containing simulated and real brain lesions not present in the training data. Performance among learning‐ and physics‐based methods was quantified in terms of structural similarity (SSIM) of the resulting image to diagnostic CT and Dice similarity metric (DSC) in lesion segmentation compared to ground truth. A pilot study was conducted involving seven subjects with CBCT images acquired during neurosurgery to assess the feasibility of DL‐Recon in clinical data. Results CBCT images reconstructed via FBP with physics‐based corrections exhibited the usual challenges to soft‐tissue contrast resolution due to image non‐uniformity, noise, and residual artifacts. GAN synthesis improved image uniformity and soft‐tissue visibility but was subject to error in the shape and contrast of simulated lesions that were unseen in training. Incorporation of aleatoric uncertainty in synthesis loss improved estimation of epistemic uncertainty, with variable brain structures and unseen lesions exhibiting higher epistemic uncertainty. The DL‐Recon approach mitigated synthesis errors while maintaining improvement in image quality, yielding 15%–22% increase in SSIM (image appearance compared to diagnostic CT) and up to 25% increase in DSC in lesion segmentation compared to FBP. Clear gains in visual image quality were also observed in real brain lesions and in clinical CBCT images. Conclusions DL‐Recon leveraged uncertainty estimation to combine the strengths of DL and physics‐based reconstruction and demonstrated substantial improvements in the accuracy and quality of intraoper...

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Combining physics‐based models with deep learning image synthesis and uncertainty in intraoperative cone‐beam CT of the brain

et al. 2023

View full text Add to dashboard Cite

show abstract

Uncertainty Estimation for Dual View X-ray Mammographic Image Registration Using Deep Ensembles

Walton,

Kim

2024

j Imaging. Inform. med.

View full text Add to dashboard Cite

Techniques are developed for generating uncertainty estimates for convolutional neural network (CNN)-based methods for registering the locations of lesions between the craniocaudal (CC) and mediolateral oblique (MLO) mammographic X-ray image views. Multi-view lesion correspondence is an important task that clinicians perform for characterizing lesions during routine mammographic exams. Automated registration tools can aid in this task, yet if the tools also provide confidence estimates, they can be of greater value to clinicians, especially in cases involving dense tissue where lesions may be difficult to see. A set of deep ensemble-based techniques, which leverage a negative log-likelihood (NLL)-based cost function, are implemented for estimating uncertainties. The ensemble architectures involve significant modifications to an existing CNN dual-view lesion registration algorithm. Three architectural designs are evaluated, and different ensemble sizes are compared using various performance metrics. The techniques are tested on synthetic X-ray data, real 2D X-ray data, and slices from real 3D X-ray data. The ensembles generate covariance-based uncertainty ellipses that are correlated with registration accuracy, such that the ellipse sizes can give a clinician an indication of confidence in the mapping between the CC and MLO views. The results also show that the ellipse sizes can aid in improving computer-aided detection (CAD) results by matching CC/MLO lesion detects and reducing false alarms from both views, adding to clinical utility. The uncertainty estimation techniques show promise as a means for aiding clinicians in confidently establishing multi-view lesion correspondence, thereby improving diagnostic capability.

show abstract

Radiation image reconstruction and uncertainty quantification using a Gaussian process prior

Lee,

Joshi,

Bandstra

et al. 2024

Sci Rep

View full text Add to dashboard Cite

We propose a complete framework for Bayesian image reconstruction and uncertainty quantification based on a Gaussian process prior (GPP) to overcome limitations of maximum likelihood expectation maximization (ML-EM) image reconstruction algorithm. The prior distribution is constructed with a zero-mean Gaussian process (GP) with a choice of a covariance function, and a link function is used to map the Gaussian process to an image. Unlike many other maximum a posteriori approaches, our method offers highly interpretable hyperparamters that are selected automatically with the empirical Bayes method. Furthermore, the GP covariance function can be modified to incorporate a priori structural priors, enabling multi-modality imaging or contextual data fusion. Lastly, we illustrate that our approach lends itself to Bayesian uncertainty quantification techniques, such as the preconditioned Crank–Nicolson method and the Laplace approximation. The proposed framework is general and can be employed in most radiation image reconstruction problems, and we demonstrate it with simulated free-moving single detector radiation source imaging scenarios. We compare the reconstruction results from GPP and ML-EM, and show that the proposed method can significantly improve the image quality over ML-EM, all the while providing greater understanding of the source distribution via the uncertainty quantification capability. Furthermore, significant improvement of the image quality by incorporating a structural prior is illustrated.

show abstract

Uncertainty quantification in medical image synthesis

Cited by 6 publications

References 121 publications

Combining physics‐based models with deep learning image synthesis and uncertainty in intraoperative cone‐beam CT of the brain

Combining physics‐based models with deep learning image synthesis and uncertainty in intraoperative cone‐beam CT of the brain

Uncertainty Estimation for Dual View X-ray Mammographic Image Registration Using Deep Ensembles

Radiation image reconstruction and uncertainty quantification using a Gaussian process prior

Contact Info

Product

Resources

About