Pseudoprospective Paraclinical Interaction of Radiology Residents With a Deep Learning System for Prostate Cancer Detection

Abstract

ObjectivesThe aim of this study was to estimate the prospective utility of a previously retrospectively validated convolutional neural network (CNN) for prostate cancer (PC) detection on prostate magnetic resonance imaging (MRI).Materials and MethodsThe biparametric (T2-weighted and diffusion-weighted) portion of clinical multiparametric prostate MRI from consecutive men included between November 2019 and September 2020 was fully automatically and individually analyzed by a CNN briefly after image acquisition (pseudoprospective design). Radiology residents performed 2 research Prostate Imaging Reporting and Data System (PI-RADS) assessments of the multiparametric dataset independent from clinical reporting (paraclinical design) before and after review of the CNN results and completed a survey. Presence of clinically significant PC was determined by the presence of an International Society of Urological Pathology grade 2 or higher PC on combined targeted and extended systematic transperineal MRI/transrectal ultrasound fusion biopsy. Sensitivities and specificities on a patient and prostate sextant basis were compared using the McNemar test and compared with the receiver operating characteristic (ROC) curve of CNN. Survey results were summarized as absolute counts and percentages.ResultsA total of 201 men were included. The CNN achieved an ROC area under the curve of 0.77 on a patient basis. Using PI-RADS ≥3-emulating probability threshold (c3), CNN had a patient-based sensitivity of 81.8% and specificity of 54.8%, not statistically different from the current clinical routine PI-RADS ≥4 assessment at 90.9% and 54.8%, respectively (P = 0.30/P = 1.0). In general, residents achieved similar sensitivity and specificity before and after CNN review. On a prostate sextant basis, clinical assessment possessed the highest ROC area under the curve of 0.82, higher than CNN (AUC = 0.76, P = 0.21) and significantly higher than resident performance before and after CNN review (AUC = 0.76 / 0.76, P ≤ 0.03). The resident survey indicated CNN to be helpful and clinically useful.ConclusionsPseudoprospective paraclinical integration of fully automated CNN-based detection of suspicious lesions on prostate multiparametric MRI was demonstrated and showed good acceptance among residents, whereas no significant improvement in resident performance was found. General CNN performance was preserved despite an observed shift in CNN calibration, identifying the requirement for continuous quality control and recalibration.

Characterization of high-grade prostate cancer at multiparametric MRI: assessment of PI-RADS version 2.1 and version 2 descriptors across 21 readers with varying experience (MULTI study)

“…Using quantitative thresholds for apparent diffusion coefficient or DCE-derived parameters may also improve prostate MRI accuracy and inter-reader agreement [16,[32][33][34], but there is still progress to be made on the reproducibility of MRI biomarkers [35][36][37][38]. Finally, assistance by Artificial Intelligence algorithms may facilitate prostate MRI reading in the future; however, conflicting results have been recently published on this matter [39][40][41][42][43][44][45].…”

An Automated Prostate-cancer Prediction System (APPS) Based on Advanced DFO-ConGA2L Model using MRI Imaging Technique

“…Positive Negative Positive TP ( 58 TNR, also known as specificity, indicates that negative tests were correctly recognized. It can be expressed mathematically as illustrated below: (13) Consequently, the positive and negative prediction values are estimated by using the following equations:…”

“…AI has many subfields, in which Machine learning (ML) is one of such subfields; it deals with algorithms that dynamically obtain information from data in order to build intelligence in systems. The following four primary groups are useful for categorize machine learning types: reinforcement learning, supervised learning, semi-supervised learning, and unsupervised learning [13]. The machine receives data from an observer who also labels the different sorts of data in supervised learning.…”

Identification of impactful imaging biomarker: Clinical applications for breast and prostate carcinoma

“…For the AI tools listed, there is currently insufficient clinical evidence that they have comparable performance to radiologists. Only individual studies currently show comparable performance [52]. However, larger validation studies, especially with multi-center data are still scarce and needed in the future.…”

“…Für die gelisteten AI-Tools gibt es derzeit keine hinreichenden klinischen Belege, dass diese vergleichbare Performance zu Radiologen besitzen. Lediglich einzelne Studien zeigen aktuell eine vergleichbare Performance 52 . Größere Validierungsstudien, insbesondere mit Multi-Center-Daten, sind jedoch noch rar und in Zukunft notwendig.…”