A non-inferiority trial comparing the accuracy of classifying urinary gram-stain findings between an artificial intelligence smartphone-based application and microbiology specialists

Abstract

Background Gram staining results interpreted by microbiology specialists (MS) were compared to a developed computer-aided diagnosis system (CAD) using artificial intelligence for interpretation. Methods Using a non-inferiority study, CAD-predicted Gram stain results, generated from images of an iPhone camera in two hospitals, were compared to those of MS, between 1 April and 31 December 2022. The prediction accuracies, classified as Class 1 based on the bacterial morphology, were the primary endpoint. The datasets were created according to the separate hospitals; 10 MS interpreted 153 images from each hospital. CAD predicted 306 images overall. Results The accuracies (95% confidence intervals) of MS and CAD predictions were 83.0% (81.6–84.3) and 87.9% (83.7–91.3), respectively; with a difference of –4.93% (–8.43 to –0.62), indicating non-inferiority of CAD. Conclusions Non-inferiority of CAD to MS predictions was demonstrated; therefore, CAD urine Gram staining can be used to select empirical antibiotics in facilities without MS.