Searching Images for Consensus

Tizhoosh, Hamid R.; Diamandis, Phedias; Campbell, Clinton J. V.; Safarpoor, Amir; Kalra, Shivam; Maleki, Danial; Riasatian, Abtin; Babaie, Morteza

doi:10.1016/j.ajpath.2021.01.015

Cited by 35 publications

(20 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is well known that the assessment of tissue images is subject to intra-and inter-observer variability [41][42][43][44][45][46][47] . This variability results from subjective biases (e.g., caused by training, specialization, and experience) but also from inherent ambiguities in the images 48,49 .…”

Section: Target Population Of Imagesmentioning

confidence: 99%

Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology

et al. 2022

View full text Add to dashboard Cite

Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recommendations are missing. A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations on compiling test datasets. We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries? The recommendations are intended to help AI developers demonstrate the utility of their products and to help pathologists and regulatory agencies verify reported performance measures. Further research is needed to formulate criteria for sufficiently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic workflows in the future.

show abstract

Section: Target Population Of Imagesmentioning

confidence: 99%

Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology

et al. 2022

View full text Add to dashboard Cite

show abstract

“…It is well known that the assessment of tissue images is subject to intra-and inter-observer variability [45][46][47][48][49][50][51]. This variability results from subjective biases (e.g., caused by training, specialization, and experience) but also from inherent ambiguities in the images [52,53].…”

Section: Target Population Of Imagesmentioning

confidence: 99%

Recommendations on test datasets for evaluating AI solutions in pathology

Homeyer,

Geißler,

Schwen

et al. 2022

Preprint

View full text Add to dashboard Cite

Arti cial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and speci c recommendations are missing.A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations for the collection of test datasets.We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in di erent countries?The recommendations are intended to help AI developers demonstrate the utility of their products and to help regulatory agencies and end users verify reported performance measures. Further research is needed to formulate criteria for su ciently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic work ows in the future.

show abstract

“…3 The evolution of AI in pathology is illuminated by earlier but analogous trends in radiology. 4 Analogies with the use of AI in reduction of interobserver variability are considered, 5 as are current strategies for reducing the need for massive annotation in machine learning through either the use of existing supervised frameworks or by exploiting hybrid models using unsupervised learning, generative models, and/or synthetic data. 6 Because of the rapid emergence of generative algorithms, a separate Review is included on generative deep learning in the pathology environment.…”

Section: Q11mentioning

confidence: 99%