Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging

Angelopoulos, Anastasios N; Kohli, Amit P; Bates, Stephen; Jordan, Michael I.; Malik, Jitendra; Alshaabi, Thayer; Upadhyayula, Srigokul; Romano, Yaniv

doi:10.48550/arxiv.2202.05265

Cited by 3 publications

(2 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, we focus specifically on the use of conformal risk control techniques [14,5,4] for the purpose of biological retrieval algorithms. Although conformal prediction has been applied in several ways to the biomedical space (see, e.g., [39,2,40,37,7,21,33,32]), we are unaware of substantial work that resembles ours. A recent work published as we were writing this manuscript, [20], leverages conformal prediction to improve a new machine learning model that the authors train, PenLight2.…”

Section: Motivation and Relevant Workmentioning

confidence: 99%

Functional protein mining with conformal guarantees

Boger,

Chithrananda,

Angelopoulos

et al. 2024

Preprint

View full text Add to dashboard Cite

1AbstractMolecular structure prediction and homology detection provide a promising path to discovering new protein function and evolutionary relationships. However, current approaches lack statistical reliability assurances, limiting their practical utility for selecting proteins for further experimental and in-silico characterization. To address this challenge, we introduce a novel approach to protein search leveraging principles from conformal prediction, offering a framework that ensures statistical guarantees with user-specified risk on outputs from any protein search model. Our method (1) lets users select any loss metric (i.e. false discovery rate) and assigns reliable functional probabilities for annotating genes of unknown function; (2) achieves state-of-the-art performance in enzyme classification without training new models; and (3) robustly and rapidly pre-filters proteins for computationally intensive structural alignment algorithms. Our framework enhances the reliability of protein homology detection and enables the discovery of new proteins with likely desirable functional properties.

show abstract

Section: Motivation and Relevant Workmentioning

confidence: 99%

Functional protein mining with conformal guarantees

Boger,

Chithrananda,

Angelopoulos

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…We also build directly on existing work involving distribution-free risk-controlling prediction sets and Learn then Test [8,3]. The LAC baseline is taken from [26], and the ordinal CDF baseline is similar to the softmax method in [5], which is in turn motivated by [15,24]. A gentle introduction to these topics and their history is available in [2], or alternatively, in [28].…”

Section: Conformal Prediction and Distribution-free Uncertainty Quant...mentioning

confidence: 99%

Improving Trustworthiness of AI Disease Severity Rating in Medical Imaging with Ordinal Conformal Prediction Sets

Lu¹,

Angelopoulos²,

Pomerantz³

2022

Preprint

View full text Add to dashboard Cite

The regulatory approval and broad clinical deployment of medical AI have been hampered by the perception that deep learning models fail in unpredictable and possibly catastrophic ways. A lack of statistically rigorous uncertainty quantification is a significant factor undermining trust in AI results. Recent developments in distribution-free uncertainty quantification present practical solutions for these issues by providing reliability guarantees for black-box models on arbitrary data distributions as formally valid finite-sample prediction intervals. Our work applies these new uncertainty quantification methods -specifically conformal prediction -to a deep-learning model for grading the severity of spinal stenosis in lumbar spine MRI. We demonstrate a technique for forming ordinal prediction sets that are guaranteed to contain the correct stenosis severity within a user-defined probability (confidence interval). On a dataset of 409 MRI exams processed by the deep-learning model, the conformal method provides tight coverage with small prediction set sizes. Furthermore, we explore the potential clinical applicability of flagging cases with high uncertainty predictions (large prediction sets) by quantifying an increase in the prevalence of significant imaging abnormalities (e.g. motion artifacts, metallic artifacts, and tumors) that could degrade confidence in predictive performance when compared to a random sample of cases.

show abstract