Reproducible And Clinically Translatable Deep Neural Networks For Cervical Screening

Abstract

Cervical cancer is a leading cause of cancer mortality, with approximately 90% of the 250,000 deaths per year occurring in low- and middle-income countries (LMIC). Secondary prevention with cervical screening involves detecting and treating precursor lesions; however, scaling screening efforts in LMIC has been hampered by infrastructure and cost constraints. Recent work has supported the development of an artificial intelligence (AI) pipeline on digital images of the cervix to achieve an accurate and reliable diagnosis of treatable precancerous lesions. In particular, WHO guidelines emphasize visual triage of women testing positive for human papillomavirus (HPV) as the primary screen, and AI could assist in this triage task. Published AI reports have exhibited overfitting, lack of portability, and unrealistic, near-perfect performance estimates. To surmount recognized issues, we implemented a comprehensive deep-learning model selection and optimization study on a large, collated, multi-institutional dataset of 9,462 women (17,013 images). We evaluated relative portability, repeatability, and classification performance. The top performing model, when combined with HPV type, achieved an area under the Receiver Operating Characteristics (ROC) curve (AUC) of 0.89 within our study population of interest, and a limited total extreme misclassification rate of 3.4%, on held-aside test sets. Our work is among the first efforts at designing a robust, repeatable, accurate and clinically translatable deep-learning model for cervical screening.

Assessing generalizability of an AI-based visual test for cervical cancer screening

“…This is achieved even in the absence of retraining, and remains relatively constant throughout incremental retraining, as Table 2 highlights. This is largely attributable to the presence of MC dropout and the dedicated optimization of repeatability as a selection criterion during model optimization (6,21,31).…”

“…In this paper, we utilized a model that we developed in a prior study, following a multistage model selection and optimization process utilizing a multi-heterogeneous dataset, henceforth referred to as "SEED" (21). The primary discernible axes of heterogeneity in this prior work included image capture device and geography.…”

“…Therefore, there is a need for a highly accurate, repeatable, low-cost, point-of-care visual screening test to triage the risks of HPV-positive individuals. To address this need, we previously conducted a comprehensive, multi-stage model selection and optimization approach, utilizing a large, collated multi-institution, multi-device, and multi-population dataset, in order to generate a diagnostic classifier, termed automated visual evaluation (AVE) that is able to classify images of the cervix into "normal", "indeterminate" (interchangeably termed as "gray zone") and "precancer/cancer" (denoted as "precancer+") categories (21). In the present work, we assess the generalizability or portability of AVE on multiple external datasets; specifically, we assess both the repeatability and classification performance, utilizing various retraining and inference strategies.…”

Generalizable deep neural networks for image quality classification of cervical images

“…1); these included different model architectures (densenet121 24 , resnet50 25 ), loss functions (standard cross-entropy, quadratic weighted kappa 26 , and mean-squared error losses) and dataset balancing strategies (balanced sampling, balanced loss). Our design choices here were informed by prior work 20 highlighting the utility of these choices across medical imaging domains, and specifically for the cervical domain.…”

“…However, numerous studies have indicated that visual evaluation by healthcare providers exhibits suboptimal accuracy and repeatability 18,19 , creating a necessity for automated tools that can more consistently evaluate cervical lesions and direct the appropriate treatment protocol. To this end, we had previously generated a multiclass diagnostic classifier able to classify the appearance of the cervix into "normal", "indeterminate" and "precancer/cancer" categories 20 .…”