Artificial Intelligent Stenosis Diagnosis in Coronary CTA: Effect on the Performance and Consistency of Readers with Less Cardiovascular Experience

Abstract

Background: To investigate the influence of artificial intelligent (AI) based on deep learning on the diagnostic performance and consistency of inexperienced cardiovascular radiologists.Methods: We enrolled 196 patents who had undergone both CCTA and invasive coronary angiography (ICA) within 6 months. Four readers with less cardiovascular experience (Reader 1 to Reader 4) and two cardiovascular radiologists (level II, Reader 5 and Reader 6) evaluated all images for ≥50% coronary artery stenosis, with ICA as the gold standard. Reader 3 and Reader 4 interpreted with aid from an AI system, and the other four readers interpreted without the AI system. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy (area under the receiver operating characteristic curve (AUC)) of six readers were calculated at the patient and vessel levels. Additionally, we evaluated interobserver consistency between Reader 1 and Reader 2, Reader 3 and Reader 4, and Reader 5 and Reader 6.Results: The AI system had 94% and 78% sensitivity at the patient and vessel levels, respectively, which were higher than Reader 5 and Reader 6. Reader 3 and Reader 4 aided by AI had a higher sensitivity (range: +7.2%~ +16.6% and +5.9%~ +16.1%, respectively) and NPV (range: +3.7%~ +13.4% and +2.7%~ +4.2%, respectively) than Reader 1 and Reader 2 without AI. There was good interobserver consistency between Reader 3 and Reader 4 in interpreting ≥50% stenosis (Kappa value= 0.75 and 0.80 at the patient and vessel levels, respectively). Only Reader 1 and Reader 2 had poor consistency (Kappa value= 0.25 and 0.37). Reader 5 and Reader 6 had moderate agreement (Kappa value= 0.55 and 0.61).Conclusions: Our study showed that using AI could effectively increase the sensitivity of inexperienced readers and significantly improve consistency in diagnosing coronary stenosis via CCTA.Trial registration: The clinical trial registration number: ChiCTR1900021867Name of registry: Diagnostic performance of artificial intelligence assisted coronary computed tomography angiography for the assessment of coronary atherosclerotic stenosis