Medical decision support system using weakly-labeled lung CT scans

Abstract

PurposeDetermination and development of an effective set of models leveraging Artificial Intelligence techniques to generate a system able to support clinical practitioners working with COVID-19 patients. It involves a pipeline including classification, lung and lesion segmentation, as well as lesion quantification of axial lung CT studies.ApproachA deep neural network architecture based on DenseNet is introduced for the classification of weakly-labeled, variable-sized (and possibly sparse) axial lung CT scans. The models are trained and tested on aggregated, publicly available data sets with over 10 categories. To further assess the models, a data set was collected from multiple medical institutions in Colombia, which includes healthy, COVID-19 and patients with other diseases. It is composed of 1,322 CT studies from a diverse set of CT machines and institutions that make over 550,000 slices. Each CT study was labeled based on a clinical test, and no per-slice annotation took place. This enabled a classification into Normal vs. Abnormal patients, and for those that were considered abnormal, an extra classification step into Abnormal (other diseases) vs. COVID-19. Additionally, the pipeline features a methodology to segment and quantify lesions of COVID-19 patients on the complete CT study, enabling easier localization and progress tracking. Moreover, multiple ablation studies were performed to appropriately assess the elements composing the classification pipeline.ResultsThe best performing lung CT study classification models achieved 0.83 accuracy, 0.79 sensitivity, 0.87 specificity, 0.82 F1 score and 0.85 precision for the Normal vs. Abnormal task. For the Abnormal vs COVID-19 task, the model obtained 0.86 accuracy, 0.81 sensitivity, 0.91 specificity, 0.84 F1 score and 0.88 precision. The ablation studies showed that using the complete CT study in the pipeline resulted in greater classification performance, restating that relevant COVID-19 patterns cannot be ignored towards the top and bottom of the lung volume.DiscussionThe lung CT classification architecture introduced has shown that it can handle weakly-labeled, variable-sized and possibly sparse axial lung studies, reducing the need for expert annotations at a per-slice level.ConclusionsThis work presents a working methodology that can guide the development of decision support systems for clinical reasoning in future interventionist or prospective studies.

Clinical decision support systems (CDSS) in assistance to COVID‐19 diagnosis: A scoping review on types and evaluation methods

“…ML was the most frequently used method to develop CDSS for COVID‐19 diagnosis. The most common ML methods in the reviewed studies were CNN 34–85 . In line with this result, other review studies also indicated CNN to be the most common method to develop a system for COVID‐19 diagnosis 23,25,27,28,30,100 .…”

“…These studies also recommended the use of CDSS in future research. Finally, 68 articles met all the inclusion criteria 5,17,34–99 . The flowchart of the selection process is shown in Figure 1.…”

“…Types of CDSS to assist in diagnosing COVID‐19 are shown in Figure 4. Most of the studies used ICDSS based on ML (nonknowledge‐based CDSS) ( n = 52 [76.5%]) 34–85 . In these studies, the most common methods for designing CDSS were CNN ( n = 33), 38,40–42,45–47,49–52,54,56–69,71,72,78,82–85 SVM ( n = 8), 35,36,39,43,44,54,56,57 RF ( n = 7), 34,35,37,39,42,44,55 and KNN ( n = 7) 36,37,39,42,43,55,56 (Table 1 and Appendix ).…”

“…Most studies used a combination of performance metrics to evaluate CDSS ( n = 40) (Appendix ). In most of these studies, accuracy ( n = 43), 35–40,42–47,49–69,71,72,76–78,80–83,85 sensitivity (recall) ( n = 35) 35–37,39,41–44,46,47,49–54,56,57,61,63–69,71,72,76,78,81–85 and F1‐score ( n = 26) 34–36,39,41–43,46,47,50,52–54,56,57,60,61,63,64,66,68,69,81,82,84,85 were used (Table 2). Other metrics for evaluating ICDSS based on ML were the Matthews correlation coefficient (MCC), 52,56,58 receiver operating characteristic (ROC) curve, 34,36,70 and so on (Appendix ).…”

“…CNN is more valuable than other methods for developing CDSS due to its excellent performance accuracy and much lower preprocessing 25,30 . For example, in the reviewed studies, CDSS based on CNN achieved a performance accuracy ranging from 75% to 99.62% 38,40–42,45–47,49–52,54,56–69,71,72,78,82–85 . Also, these studies indicated that CDSS based on CNN could be effective in detecting COVID‐19, assisting domain specialists, physicians, and radiologists in the decision‐making process, and enhancing radiologists' working productivity 25,30,42,66,87 It is challenging to create a CDSS using CNN due to the scarcity of big data and low quality of data sources 25 .…”