Assessing the Performance of Artificial Intelligence Assistance for Prostate MRI: A Two‐Center Study Involving Radiologists With Different Experience Levels

Abstract

BackgroundArtificial intelligence (AI) assistance may enhance radiologists' performance in detecting clinically significant prostate cancer (csPCa) on MRI. Further validation is needed for radiologists with different experiences.PurposeTo assess the performance of experienced and less‐experienced radiologists in detecting csPCa, with and without AI assistance.Study TypeRetrospective.PopulationNine hundred patients who underwent prostate MRI and biopsy (median age 67 years; 356 with csPCa and 544 with non‐csPCa).Field Strength/Sequence3‐T and 1.5‐T, diffusion‐weighted imaging using a single‐shot gradient echo‐planar sequence, turbo spin echo T2‐weighted image.AssessmentCsPCa regions based on biopsy results served as the reference standard. Ten less‐experienced (<500 prostate MRIs) and six experienced (>1000 prostate MRIs) radiologists reviewed each case twice using Prostate Imaging Reporting and Data System v2.1, with and without AI, separated by 4‐week intervals. Cases were equally distributed among less‐experienced radiologists, and 90 cases were randomly assigned to each experienced radiologist. Reading time and diagnostic confidence were assessed.Statistical TestsArea under the curve (AUC), sensitivity, specificity, reading time, and diagnostic confidence were compared using the DeLong test, Chi‐squared test, Fisher exact test, or Wilcoxon rank‐sum test between the two sessions. A P‐value <0.05 was considered significant. Adjusting threshold using Bonferroni correction was performed for multiple comparisons.ResultsFor less‐experienced radiologists, AI assistance significantly improved lesion‐level sensitivity (0.78 vs. 0.88), sextant‐level AUC (0.84 vs. 0.93), and patient‐level AUC (0.84 vs. 0.89). For experienced radiologists, AI assistance only improved sextant‐level AUC (0.82 vs. 0.91). AI assistance significantly reduced median reading time (250 s [interquartile range, IQR: 157, 402] vs. 130 s [IQR: 88, 209]) and increased diagnostic confidence (5 [IQR: 4, 5] vs. 5 [IQR: 4, 5]) irrespective of experience and enhanced consistency among experienced radiologists (Fleiss κ: 0.53 vs. 0.61).Data ConclusionAI‐assisted reading improves the performance of detecting csPCa on MRI, particularly for less‐experienced radiologists.Evidence Level3Technical EfficacyStage 2