How to deal with Uncertainty in Machine Learning for Medical Imaging?

Gillmann, Christina; Saur, Dorothee; Scheuermann, Gerik

doi:10.1109/trex53765.2021.00014

Cited by 14 publications

(7 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous studies have demonstrated the usefulness of estimating uncertainty in computational pathology, in which a common strategy involves identifying the most uncertain predictions for manual review by medical professionals, allowing them to focus on challenging cases 23 , 24 . Similarly, uncertainty heatmaps can be generated and superimposed on the original image for visual inspection 25 , 26 . The latter approach can be valuable during the algorithm development phase, but it is impractical for pathologists under time constraints in clinical production.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Benefits of spatial uncertainty aggregation for segmentation in digital pathology

Pocevičiūtė,

Eilertsen,

Lundström

2024

J. Med. Imag.

View full text Add to dashboard Cite

.PurposeUncertainty estimation has gained significant attention in recent years for its potential to enhance the performance of deep learning (DL) algorithms in medical applications and even potentially address domain shift challenges. However, it is not straightforward to incorporate uncertainty estimation with a DL system to achieve a tangible positive effect. The objective of our work is to evaluate if the proposed spatial uncertainty aggregation (SUA) framework may improve the effectiveness of uncertainty estimation in segmentation tasks. We evaluate if SUA boosts the observed correlation between the uncertainty estimates and false negative (FN) predictions. We also investigate if the observed benefits can translate to tangible improvements in segmentation performance.ApproachOur SUA framework processes negative prediction regions from a segmentation algorithm and detects FNs based on an aggregated uncertainty score. It can be utilized with many existing uncertainty estimation methods to boost their performance. We compare the SUA framework with a baseline of processing individual pixel’s uncertainty independently.ResultsThe results demonstrate that SUA is able to detect FN regions. It achieved Fβ=0.5 of 0.92 on the in-domain and 0.85 on the domain-shift test data compared with 0.81 and 0.48 achieved by the baseline uncertainty, respectively. We also demonstrate that SUA yields improved general segmentation performance compared with utilizing the baseline uncertainty.ConclusionsWe propose the SUA framework for incorporating and utilizing uncertainty estimates for FN detection in DL segmentation algorithms for histopathology. The evaluation confirms the benefits of our approach compared with assessing pixel uncertainty independently.

show abstract

Section: Related Workmentioning

confidence: 99%

“…23,24 Similarly, uncertainty heatmaps can be generated and superimposed on the original image for visual inspection. 25,26 The latter approach can be valuable during the algorithm development phase, but it is impractical for pathologists under time constraints in clinical production.…”

Section: Related Workmentioning

confidence: 99%

Benefits of spatial uncertainty aggregation for segmentation in digital pathology

Pocevičiūtė,

Eilertsen,

Lundström

2024

J. Med. Imag.

View full text Add to dashboard Cite

show abstract

“…Alternatively, other researchers have argued for a manual visual review of generated uncertainty heatmaps that are overlayed on the original input image. 19 Such an approach can bring much value during the development stage of an algorithm, but is problematic to introduce into pathologists' review in clinical production due to the high time pressure.…”

Section: Related Workmentioning

confidence: 99%

Spatial uncertainty aggregation for false negatives detection in breast cancer metastases segmentation

Pocevičiūtė

Eilertsen

Lundström

2023

Medical Imaging 2023: Digital and Computational Pathology

View full text Add to dashboard Cite

Computational pathology, a developing area of primarily deep learning (DL) solutions aiming to aid pathologists at their daily tasks, has shown promising results in research settings. In recent years, uncertainty estimation has gained substantial recognition as having high potential to bring value to DL algorithms for medical applications. But it is not trivial how to incorporate it with a DL system to obtain a real positive impact. In this work we propose a framework to spatially aggregated epistemic uncertainty in order to detect false negatives produced by a segmentation algorithm of breast cancer metastases. We show a strong correlation between the false negative segmentation areas and the aggregated uncertainty values. Furthermore, the results include examples of reducing false negatives, where the uncertainty approach led to detection of some tumour metastases that had been missed.

show abstract

“…In machine learning, both the uncertainty inherent in the predictive model and the uncertainty inherent in the learning data themselves affect the ambiguity of predictions. In the development of diagnostic support systems, there are two primary objectives related to uncertainty assessment [8,9]. One objective is to quantitatively characterize the statistical fluctuations inherent in the predictive models themselves, while the other is to evaluate the uncertainty associated with the labels provided by human observers, which serve as the ground truth for model training.…”

Section: Introductionmentioning

confidence: 99%

Quantitative evaluation model of variable diagnosis for chest X-ray images using deep learning

Nakagawa,

Ono,

Hakamata

et al. 2024

PLOS Digit Health

View full text Add to dashboard Cite

The purpose of this study is to demonstrate the use of a deep learning model in quantitatively evaluating clinical findings typically subject to uncertain evaluations by physicians, using binary test results based on routine protocols. A chest X-ray is the most commonly used diagnostic tool for the detection of a wide range of diseases and is generally performed as a part of regular medical checkups. However, when it comes to findings that can be classified as within the normal range but are not considered disease-related, the thresholds of physicians’ findings can vary to some extent, therefore it is necessary to define a new evaluation method and quantify it. The implementation of such methods is difficult and expensive in terms of time and labor. In this study, a total of 83,005 chest X-ray images were used to diagnose the common findings of pleural thickening and scoliosis. A novel method for quantitatively evaluating the probability that a physician would judge the images to have these findings was established. The proposed method successfully quantified the variation in physicians’ findings using a deep learning model trained only on binary annotation data. It was also demonstrated that the developed method could be applied to both transfer learning using convolutional neural networks for general image analysis and a newly learned deep learning model based on vector quantization variational autoencoders with high correlations ranging from 0.89 to 0.97.

show abstract

How to deal with Uncertainty in Machine Learning for Medical Imaging?

Cited by 14 publications

References 44 publications

Benefits of spatial uncertainty aggregation for segmentation in digital pathology

Benefits of spatial uncertainty aggregation for segmentation in digital pathology

Spatial uncertainty aggregation for false negatives detection in breast cancer metastases segmentation

Quantitative evaluation model of variable diagnosis for chest X-ray images using deep learning

Contact Info

Product

Resources

About