Differing Benefits of Artificial Intelligence-based Computer-aided Diagnosis (AI-CAD) for Breast US According to Workflow and Experience Level

Abstract

Background: To evaluate how artificial intelligence-based computer-assisted diagnosis (AI-CAD) for breast ultrasound (US) influences diagnostic performance and agreements between radiologists with varying experience levels in different workflows.Methods: From Apr 2017 to Jun 2018, images of 492 breast lesions (200 malignant and 292 benign masses) in 472 women were included. Six radiologists (3 inexperienced with < 1 year of experience, 3 experienced with 10-15 years of experience) individually reviewed US images with and without the aid of AI-CAD, first in sequential and then in independent reading. Diagnostic performances and interobserver agreements were calculated and compared between radiologists and AI-CAD. Results: After implementing AI-CAD, specificity, PPV and accuracy significantly improved, regardless of experience and workflow (all P<0.001, respectively). Overall area under the receiving operator characteristics curve (AUC) significantly increased in independent reading, but only for inexperienced radiologists. Agreements for BI-RADS descriptors generally increased when AI-CAD was used (κ=0.29-0.63 to 0.35-0.73). Inexperienced radiologists tended to concede to AI-CAD results more easily than experienced radiologists, especially in independent reading (P<0.001). Conversion rates for final assessment changes from BI-RADS 2 or 3 to BI-RADS higher than 4a or vice versa were also significantly higher in independent reading than sequential reading (overall: 15.8% and 6.2%, respectively, P<0.001) for both inexperienced and experienced radiologists. Conclusions: Using AI-CAD to interpret breast US improves the specificity, PPV and accuracy of radiologists regardless of experience level. AI-CAD may work better in independent reading to improve diagnostic performance and agreements between radiologists, especially for inexperienced radiologists.Trial registration: retrospectively registered