Practical guidance on artificial intelligence for health-care data

Ghassemi, Marzyeh; Naumann, Tristan; Schulam, Peter; Beam, Andrew L.; Chen, Irene Y.; Ranganath, Rajesh

doi:10.1016/s2589-7500(19)30084-6

Cited by 81 publications

(75 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Guidelines for designing and reporting traditional clinical trials are now also being specialised for AI-based interventions 47 . This has been accompanied by a recent surge in discussion among the medical community about the opportunities and, crucially, the risks of deploying such tools in clinical practice [48][49][50][51][52][53][54][55] . Most of the apprehension revolves around the external validity of these predictive models, i.e.…”

Section: Discussionmentioning

confidence: 99%

Causality matters in medical imaging

2020

View full text Add to dashboard Cite

Causal reasoning can shed new light on the major challenges in machine learning for medical imaging: scarcity of high-quality annotated data and mismatch between the development dataset and the target environment. A causal perspective on these issues allows decisions about data collection, annotation, preprocessing, and learning strategies to be made and scrutinized more transparently, while providing a detailed categorisation of potential biases and mitigation techniques. Along with worked clinical examples, we highlight the importance of establishing the causal relationship between images and their annotations, and offer stepby-step recommendations for future studies.

show abstract

Section: Discussionmentioning

confidence: 99%

Causality matters in medical imaging

2020

View full text Add to dashboard Cite

show abstract

“…2 Often the data itself introduces its own modelling challenges. 3,4 Artifacts could cause a model to learn spurious rules, for example, such as the skin cancer algorithm that associates suspicious lesions with the surgical skin markers that surround them. 5 Data quality might drift over time (for example, with changing equipment or operators), confounding an analysis that fails to account for these changes.…”

Section: Turning the Crank For Machine Learning: Ease At What Expense?mentioning

confidence: 99%

Turning the crank for machine learning: ease, at what expense?

Pollard

Chen

Wiens

et al. 2019

The Lancet Digital Health

Self Cite

View full text Add to dashboard Cite

“…Therefore, the data collection process, which is mostly carried out in a retrospective manner, is prone to various selection biases, notably spectrum bias and unnatural prevalence [12,31,34]. Additionally, there is often substantial heterogeneity in patient characteristics, equipment, facilities, and practice pattern according to hospitals, physicians, time periods, and governmental health policies [3,35]. These factors, combined with overfitting and strong data dependency of DL, can substantially undermine the generalizability and usability of DL algorithms for providing diagnosis in clinical practice [3,8,9].…”

Section: Plos Onementioning

confidence: 99%

Inconsistency in the use of the term “validation” in studies reporting the performance of deep learning algorithms in providing diagnosis from medical imaging

Kim

Jang

et al. 2020

PLoS ONE

View full text Add to dashboard Cite

Background The development of deep learning (DL) algorithms is a three-step process-training, tuning, and testing. Studies are inconsistent in the use of the term "validation", with some using it to refer to tuning and others testing, which hinders accurate delivery of information and may inadvertently exaggerate the performance of DL algorithms. We investigated the extent of inconsistency in usage of the term "validation" in studies on the accuracy of DL algorithms in providing diagnosis from medical imaging. Methods and findings We analyzed the full texts of research papers cited in two recent systematic reviews. The papers were categorized according to whether the term "validation" was used to refer to tuning alone, both tuning and testing, or testing alone. We analyzed whether paper characteristics (i.e., journal category, field of study, year of print publication, journal impact factor [JIF], and nature of test data) were associated with the usage of the terminology using multivariable logistic regression analysis with generalized estimating equations. Of 201 papers published in 125 journals, 118 (58.7%), 9 (4.5%), and 74 (36.8%) used the term to refer to tuning alone, both tuning and testing, and testing alone, respectively. A weak association was noted between higher JIF and using the term to refer to testing (i.e., testing alone or both tuning and testing) instead of tuning alone (vs. JIF <5; JIF 5 to 10: adjusted odds ratio 2.11, P = 0.042; JIF >10: adjusted odds ratio 2.41, P = 0.089). Journal category, field of study, year of print publication, and nature of test data were not significantly associated with the terminology usage.

show abstract

Practical guidance on artificial intelligence for health-care data

Cited by 81 publications

References 10 publications

Causality matters in medical imaging

Causality matters in medical imaging

Turning the crank for machine learning: ease, at what expense?

Inconsistency in the use of the term “validation” in studies reporting the performance of deep learning algorithms in providing diagnosis from medical imaging

Contact Info

Product

Resources

About