The Need for Medical Artificial Intelligence That Incorporates Prior Images

Acosta, Julián; Falcone, Guido J; Rajpurkar, Pranav

doi:10.1148/radiol.212830

Cited by 24 publications

(17 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In confirmation of our hypothesis that this unique dataset could potentially lead to the discovery of novel biomarkers, we intuitively found that the monthly rate of change in intracranial tumors is highly predictive of survival. As noted in a recent editorial in Radiology by JN Acosta et al 5 , there is a need for medical artificial intelligence (AI) that incorporates prior imaging which is analogous to how physicians perceive and utilize medical imaging. Thinking about longitudinal imaging also empowers research to develop metrics that are clinically significant (e.g.…”

Section: Discussion Limitations and Future Workmentioning

confidence: 99%

“…With the advancement of modern electronic health records (EHRs) and picture and archiving systems (PACs), it is possible to study the detailed, real-world outcomes of cancer patients and their tumors over the entire course of care. 4,5 The complexity of metastatic cancer makes this challenging, as tracking multiple metastatic lesions and diverse treatments spanning medications, radiation, and surgery is a formidable challenge that existing analytical systems struggle to perform. In the specific case of the imaging of brain metastases, MRI is the gold standard for tracking metastatic cancer.…”

Section: Introductionmentioning

confidence: 99%

“…Datacentric AI approaches emphasize that novel datasets drive algorithmic advances, and can lead to the development of entirely new classes of DNNs by incorporating new, diverse datatypes (genomic, imaging, clinical) or longitudinal information. 4,5 For biomedical imaging, longitudinal information represents a unique departure due to the significant change in its extraction, transformation, and loading which resembles sparsely sampled video data due to the inclusion of an additional dimension of time across irregular intervals. 22 Firstorder statistics derived from the dynamics of patients and tumors changing over time could provide a source of biomarkers for treatment progression, response, and survival.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Longitudinal deep neural networks for assessing metastatic brain cancer on a massive open benchmark.

Oermann

Link

Schnurman

et al. 2023

Preprint

View full text Add to dashboard Cite

The detection and tracking of metastatic cancer over the lifetime of a patient remains a major challenge in clinical trials and real-world care. 1–3 Recent advances in deep learning combined with massive, real-world datasets may enable the development of tools that can address this challenge. We present our work with the NYUMets Project to develop NYUMets-Brain and a novel longitudinal deep neural network (DNN), segmentation-through-time (STT). NYUMets-Brain is the world's largest, longitudinal, real-world dataset of cancer consisting of the imaging, clinical follow-up, and medical management of 1,429 patients with an average of six MRI studies obtained over 17 months per patient. The dataset includes over 2,367 expert segmentations of metastatic brain tumors, and 81,562 medical prescriptions. Using this dataset we developed Segmentation Through Time (STT), a deep neural network (DNN) which explicitly utilizes the longitudinal structure of the data and obtained state of the art results at tumor segmentation and detection of small (< 10 mm3) metastases. We also demonstrate that longitudinal measurements to assess the monthly rate of change of brain metastases over time are strongly predictive of overall survival (HR 1.27, 95%CI 1.18–1.38). We are releasing the entire dataset, codebase, trained model weights, and an interface for dataset access for other cancer researchers to build upon these results and to serve as a public benchmark. Massive real-world datasets and public benchmarks such as NYUMets-Brain may enable the tracking and detection of metastatic brain cancer, and be broadly applicable to advancing the development of AI models in other types of metastatic cancer as well.

show abstract

Section: Discussion Limitations and Future Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Longitudinal deep neural networks for assessing metastatic brain cancer on a massive open benchmark.

Oermann

Link

Schnurman

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…More than 300 AI-SaMDs have been approved by the U.S. FDA so far, and clinical applications are being explored, focused on medical image (radiological, endoscopic, ultrasound, etc.) analysis [ 98 , 99 ]. Additionally, AI is also being introduced for analysis of omics, and medical information and research papers [ 100 – 105 ].…”

Section: Medical Applications Of Computer-aided Diagnosis Support And...mentioning

confidence: 99%

“…Second, to date, medical image analysis has been the leading medical AI research and development method, and most AI-based medical device programs approved by the FDA are also targeted at medical image analysis [ 30 , 31 , 99 , 180 , 181 ]. Compared to medical image analysis, the introduction of AI into omics analysis has not progressed.…”

Section: Current Challenges and Possible Future Ai-based Mtbsmentioning

confidence: 99%

Introducing AI to the molecular tumor board: one direction toward the establishment of precision medicine using large-scale cancer clinical and biological information

et al. 2022

View full text Add to dashboard Cite

Since U.S. President Barack Obama announced the Precision Medicine Initiative in his New Year’s State of the Union address in 2015, the establishment of a precision medicine system has been emphasized worldwide, particularly in the field of oncology. With the advent of next-generation sequencers specifically, genome analysis technology has made remarkable progress, and there are active efforts to apply genome information to diagnosis and treatment. Generally, in the process of feeding back the results of next-generation sequencing analysis to patients, a molecular tumor board (MTB), consisting of experts in clinical oncology, genetic medicine, etc., is established to discuss the results. On the other hand, an MTB currently involves a large amount of work, with humans searching through vast databases and literature, selecting the best drug candidates, and manually confirming the status of available clinical trials. In addition, as personalized medicine advances, the burden on MTB members is expected to increase in the future. Under these circumstances, introducing cutting-edge artificial intelligence (AI) technology and information and communication technology to MTBs while reducing the burden on MTB members and building a platform that enables more accurate and personalized medical care would be of great benefit to patients. In this review, we introduced the latest status of elemental technologies that have potential for AI utilization in MTB, and discussed issues that may arise in the future as we progress with AI implementation.

show abstract