Performance of Deep Learning and Genitourinary Radiologists in Detection of Prostate Cancer Using 3‐T Multiparametric Magnetic Resonance Imaging

Cao, Ruiming; Zhong, Xinran; Afshari, Sohrab; Felker, Ely; Suvannarerg, Voraparee; Tubtawee, Teeravut; Vangala, Sitaram; Scalzo, Fabien; Raman, Steven S.; Sung, Kyunghyun

doi:10.1002/jmri.27595

Cited by 27 publications

(33 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Three out of 12 DL studies (25%) [ 37 – 39 ] that underwent quality screening using CLAIM failed at least three pre-identified mandatory criteria, with 2/12 [ 40 , 41 ] failing two, and 2/12 [ 42 , 43 ] failing just one criterion. Four of the seven rejected papers (57%) [ 37 – 39 , 43 ] did not describe data processing steps in sufficient detail (Q9), 4/7 [ 38 – 40 , 42 ] did not explain the exact method of selecting the final model (Q26), and 3/7 [ 38 , 40 , 41 ] failed to provide enough details on training approach (Q25). Following the subsequent full CLAIM assessment of the remaining five papers, we found that none of them reported the following items: selection of data subsets (Q10), robustness or sensitivity analysis (Q30), validation or testing on external data (Q32), and failure analysis of incorrectly classified cases (Q37).…”

Section: Resultsmentioning

confidence: 99%

Comparative performance of fully-automated and semi-automated artificial intelligence methods for the detection of clinically significant prostate cancer on MRI: a systematic review

et al. 2022

View full text Add to dashboard Cite

Objectives We systematically reviewed the current literature evaluating the ability of fully-automated deep learning (DL) and semi-automated traditional machine learning (TML) MRI-based artificial intelligence (AI) methods to differentiate clinically significant prostate cancer (csPCa) from indolent PCa (iPCa) and benign conditions. Methods We performed a computerised bibliographic search of studies indexed in MEDLINE/PubMed, arXiv, medRxiv, and bioRxiv between 1 January 2016 and 31 July 2021. Two reviewers performed the title/abstract and full-text screening. The remaining papers were screened by four reviewers using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) for DL studies and Radiomics Quality Score (RQS) for TML studies. Papers that fulfilled the pre-defined screening requirements underwent full CLAIM/RQS evaluation alongside the risk of bias assessment using QUADAS-2, both conducted by the same four reviewers. Standard measures of discrimination were extracted for the developed predictive models. Results 17/28 papers (five DL and twelve TML) passed the quality screening and were subject to a full CLAIM/RQS/QUADAS-2 assessment, which revealed a substantial study heterogeneity that precluded us from performing quantitative analysis as part of this review. The mean RQS of TML papers was 11/36, and a total of five papers had a high risk of bias. AUCs of DL and TML papers with low risk of bias ranged between 0.80–0.89 and 0.75–0.88, respectively. Conclusion We observed comparable performance of the two classes of AI methods and identified a number of common methodological limitations and biases that future studies will need to address to ensure the generalisability of the developed models.

show abstract

Section: Resultsmentioning

confidence: 99%

Comparative performance of fully-automated and semi-automated artificial intelligence methods for the detection of clinically significant prostate cancer on MRI: a systematic review

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Algorithm training and testing were performed retrospectively using cross-validation without an independent test set and application to new cases. In a recent study, Cao et al 42 trained a FocalNet to automatically detect prostate lesions with a confidence score achieving comparable results to experienced radiologists only in a high-sensitivity or high-specificity setting. Most recently, Li et al 40 trained a fully automatic system for lesion detection and classification and demonstrated an improvement in radiologist performance with AI support; however, the 2 readers possessed comparably low sensitivity of 59% and 79%, respectively, before AI review.…”

Section: Discussionmentioning

confidence: 99%

Pseudoprospective Paraclinical Interaction of Radiology Residents With a Deep Learning System for Prostate Cancer Detection

et al. 2022

View full text Add to dashboard Cite

ObjectivesThe aim of this study was to estimate the prospective utility of a previously retrospectively validated convolutional neural network (CNN) for prostate cancer (PC) detection on prostate magnetic resonance imaging (MRI).Materials and MethodsThe biparametric (T2-weighted and diffusion-weighted) portion of clinical multiparametric prostate MRI from consecutive men included between November 2019 and September 2020 was fully automatically and individually analyzed by a CNN briefly after image acquisition (pseudoprospective design). Radiology residents performed 2 research Prostate Imaging Reporting and Data System (PI-RADS) assessments of the multiparametric dataset independent from clinical reporting (paraclinical design) before and after review of the CNN results and completed a survey. Presence of clinically significant PC was determined by the presence of an International Society of Urological Pathology grade 2 or higher PC on combined targeted and extended systematic transperineal MRI/transrectal ultrasound fusion biopsy. Sensitivities and specificities on a patient and prostate sextant basis were compared using the McNemar test and compared with the receiver operating characteristic (ROC) curve of CNN. Survey results were summarized as absolute counts and percentages.ResultsA total of 201 men were included. The CNN achieved an ROC area under the curve of 0.77 on a patient basis. Using PI-RADS ≥3-emulating probability threshold (c3), CNN had a patient-based sensitivity of 81.8% and specificity of 54.8%, not statistically different from the current clinical routine PI-RADS ≥4 assessment at 90.9% and 54.8%, respectively (P = 0.30/P = 1.0). In general, residents achieved similar sensitivity and specificity before and after CNN review. On a prostate sextant basis, clinical assessment possessed the highest ROC area under the curve of 0.82, higher than CNN (AUC = 0.76, P = 0.21) and significantly higher than resident performance before and after CNN review (AUC = 0.76 / 0.76, P ≤ 0.03). The resident survey indicated CNN to be helpful and clinically useful.ConclusionsPseudoprospective paraclinical integration of fully automated CNN-based detection of suspicious lesions on prostate multiparametric MRI was demonstrated and showed good acceptance among residents, whereas no significant improvement in resident performance was found. General CNN performance was preserved despite an observed shift in CNN calibration, identifying the requirement for continuous quality control and recalibration.

show abstract

“…target Gleason score, growth patterns, surface area, volume) need to be accurately established to correctly rate a given algorithm's underlying dataset relative to the natural history of the targeted prostate cancer. These ground truth data are difficult to compile, as digital annotations of prostate MRI scans are rarely performed in clinical practice, and histopathologic correlation with whole gland sampling [30] for MRI verification does not happen in the clinical routine. For example, men with negative scans may not undergo biopsy and men with positive scans may only undergo targeted biopsies.…”

Section: Data Science Prerequisitesmentioning

confidence: 99%

ESUR/ESUI position paper: developing artificial intelligence for precision diagnosis of prostate cancer using magnetic resonance imaging

et al. 2021

View full text Add to dashboard Cite

Artificial intelligence developments are essential to the successful deployment of community-wide, MRI-driven prostate cancer diagnosis. AI systems should ensure that the main benefits of biopsy avoidance are delivered while maintaining consistent high specificities, at a range of disease prevalences. Since all current artificial intelligence / computer-aided detection systems for prostate cancer detection are experimental, multiple developmental efforts are still needed to bring the vision to fruition. Initial work needs to focus on developing systems as diagnostic supporting aids so their results can be integrated into the radiologists’ workflow including gland and target outlining tasks for fusion biopsies. Developing AI systems as clinical decision-making tools will require greater efforts. The latter encompass larger multicentric, multivendor datasets where the different needs of patients stratified by diagnostic settings, disease prevalence, patient preference, and clinical setting are considered. AI-based, robust, standard operating procedures will increase the confidence of patients and payers, thus enabling the wider adoption of the MRI-directed approach for prostate cancer diagnosis. Key Points • AI systems need to ensure that the benefits of biopsy avoidance are delivered with consistent high specificities, at a range of disease prevalence. • Initial work has focused on developing systems as diagnostic supporting aids for outlining tasks, so they can be integrated into the radiologists’ workflow to support MRI-directed biopsies. • Decision support tools require a larger body of work including multicentric, multivendor studies where the clinical needs, disease prevalence, patient preferences, and clinical setting are additionally defined.

show abstract

Performance of Deep Learning and Genitourinary Radiologists in Detection of Prostate Cancer Using 3‐T Multiparametric Magnetic Resonance Imaging

Cited by 27 publications

References 26 publications

Comparative performance of fully-automated and semi-automated artificial intelligence methods for the detection of clinically significant prostate cancer on MRI: a systematic review

Comparative performance of fully-automated and semi-automated artificial intelligence methods for the detection of clinically significant prostate cancer on MRI: a systematic review

Pseudoprospective Paraclinical Interaction of Radiology Residents With a Deep Learning System for Prostate Cancer Detection

ESUR/ESUI position paper: developing artificial intelligence for precision diagnosis of prostate cancer using magnetic resonance imaging

Contact Info

Product

Resources

About