Improving the quality of machine learning in health applications and clinical research

Mateen, Bilal A.; Liley, James; Denniston, Alastair K; Holmes, Chris; Vollmer, Sebastian J.

doi:10.1038/s42256-020-00239-1

Cited by 62 publications

(44 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CT images in particular might not accurately reflect the biological tumour characteristics due to insu cient resolution, sensitivity to acquisition parameters and noise 48,49 , as well as the source of image contrast, which is essentially electron density of the tissue which demonstrates little texture at current image scales. This highlights the broader need of greater collaboration between ML researchers, clinicians and physicists, also in data selection and experiment design -with reciprocal feedback 50,51 .…”

Section: Discussionmentioning

confidence: 99%

A Machine Learning Challenge for Prognostic Modelling in Head and Neck Cancer Using Multi-modal Data

Haibe‐Kains¹,

Kazmierski²,

Welch³

et al. 2021

Preprint

View full text Add to dashboard Cite

Accurate prognosis for an individual patient is a key component of precision oncology. Recent advances in machine learning have enabled the development of models using a wider range of data, including imaging. Radiomics aims to extract quantitative predictive and prognostic biomarkers from routine medical imaging, but evidence for computed tomography radiomics for prognosis remains inconclusive. We have conducted an institutional machine learning challenge to develop an accurate model for overall survival prediction in head and neck cancer using clinical data etxracted from electronic medical records and pre-treatment radiological images, as well as to evaluate the true added benefit of radiomics for head and neck cancer prognosis. Using a large, retrospective dataset of 2,552 patients and a rigorous evaluation framework, we compared 12 different submissions using imaging and clinical data, separately or in combination. The winning approach used non-linear, multitask learning on clinical data and tumour volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction and outperforming models relying on clinical data only, engineered radiomics and deep learning. Combining all submissions in an ensemble model resulted in improved accuracy, with the highest gain from a image-based deep learning model. Our results show the potential of machine learning and simple, informative prognostic factors in combination with large datasets as a tool to guide personalized cancer care.

show abstract

Section: Discussionmentioning

confidence: 99%

A Machine Learning Challenge for Prognostic Modelling in Head and Neck Cancer Using Multi-modal Data

Haibe‐Kains¹,

Kazmierski²,

Welch³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, in spite of the huge number of published studies, most applications still fail to enter routine practice, even if they perform well in experiments and clinical trials (see reviews in [ 2 , 90 ] and, specifically with regard to neurology, in [ 47 ]). No study has identified methods pertaining to predicting the course of MS with performances usable in the clinics.…”

Section: Discussionmentioning

confidence: 99%

Machine Learning Use for Prognostic Purposes in Multiple Sclerosis

Seccia

Romano

Salvetti

et al. 2021

Life

View full text Add to dashboard Cite

The course of multiple sclerosis begins with a relapsing-remitting phase, which evolves into a secondarily progressive form over an extremely variable period, depending on many factors, each with a subtle influence. To date, no prognostic factors or risk score have been validated to predict disease course in single individuals. This is increasingly frustrating, since several treatments can prevent relapses and slow progression, even for a long time, although the possible adverse effects are relevant, in particular for the more effective drugs. An early prediction of disease course would allow differentiation of the treatment based on the expected aggressiveness of the disease, reserving high-impact therapies for patients at greater risk. To increase prognostic capacity, approaches based on machine learning (ML) algorithms are being attempted, given the failure of other approaches. Here we review recent studies that have used clinical data, alone or with other types of data, to derive prognostic models. Several algorithms that have been used and compared are described. Although no study has proposed a clinically usable model, knowledge is building up and in the future strong tools are likely to emerge.

show abstract

“…However, there is a growing awareness in the community that the presence of different sources of bias significantly decreases the overall generalisation ability of the models, leading to overestimated model performance reported in internal validation compared to evaluation on independent test data ( Soneson, Gerster, Delorenzi, 2014 , Cohen, Hashir, Brooks, Bertrand , Zech, Badgeley, Liu, Costa, Titano, Oermann, 2018 , Maguolo, Nanni ). In addition, numerous journal editorials are calling for better development, evaluation and reporting practices of machine learning models aimed for clinical application ( Mateen, Liley, Denniston, Holmes, Vollmer, 2020 , Nagendran, Chen, Lovejoy, Gordon, Komorowski, Harvey, Topol, Ioannidis, Collins, Maruthappu, 2020 , Campbell, Lee, Abrmoff, Keane, Ting, Lum, Chiang, 2020 , Health, 2020 , O’Reilly-Shah, Gentry, Walters, Zivot, Anderson, Tighe, 2020 , Health, 2019 , Stevens, Mortazavi, Deo, Curtis, Kao, 2020 ). Underneath, there are growing concerns about ethics and the risk of harmful outcomes of using AI in medical applications ( Campolo, Sanfilippo, Whittaker, Crawford, 2018 , Geis, Brady, Wu, Spencer, Ranschaert, Jaremko, Langer, Borondy Kitts, Birch, Shields, van den Hoven van Genderen, Kotter, Wawira Gichoya, Cook, Morgan, Tang, Safdar, Kohli, 2019 , Brady, Neri, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

Public Covid-19 X-ray datasets and their impact on model bias – A systematic review of a significant problem

Cruz

Bossa

Sölter

et al. 2021

Medical Image Analysis

View full text Add to dashboard Cite

show abstract

Improving the quality of machine learning in health applications and clinical research

Cited by 62 publications

References 14 publications

A Machine Learning Challenge for Prognostic Modelling in Head and Neck Cancer Using Multi-modal Data

A Machine Learning Challenge for Prognostic Modelling in Head and Neck Cancer Using Multi-modal Data

Machine Learning Use for Prognostic Purposes in Multiple Sclerosis

Public Covid-19 X-ray datasets and their impact on model bias – A systematic review of a significant problem

Contact Info

Product

Resources

About