3D Deep Learning from CT Scans Predicts Tumor Invasiveness of Subcentimeter Pulmonary Adenocarcinomas

Abstract

Identification of early-stage pulmonary adenocarcinomas before surgery, especially in cases of subcentimeter cancers, would be clinically important and could provide guidance to clinical decision making. In this study, we developed a deep learning system based on 3D convolutional neural networks and multitask learning, which automatically predicts tumor invasiveness, together with 3D nodule segmentation masks. The system processes a 3D nodule-centered patch of preprocessed CT and learns a deep representation of a given nodule without the need for any additional information. A dataset of 651 nodules with manually segmented voxel-wise masks and pathological labels of atypical adenomatous hyperplasia (AAH), adenocarcinomas in situ (AIS), minimally invasive adenocarcinoma (MIA), and invasive pulmonary adenocarcinoma (IA) was used in this study. We trained and validated our deep learning system on 523 nodules and tested its performance on 128 nodules. An observer study with 2 groups of radio-logists, 2 senior and 2 junior, was also investigated. We merged AAH and AIS into one single category AAH-AIS, comprising a 3-category classification in our study. The proposed deep learning system achieved better classification performance than the radiologists; in terms of 3-class weighted average F1 score, the model achieved 63.3% while the radiologists achieved 55.6%, 56.6%, 54.3%, and 51.0%, respectively. These results suggest that deep learning methods improve the yield of discriminative results and hold promise in the CADx application domain, which could help doctors work efficiently and facilitate the application of precision medicine.Significance: Machine learning tools are beginning to be implemented for clinical applications. This study represents an important milestone for this emerging technology, which could improve therapy selection for patients with lung cancer.

Toward automatic prediction of EGFR mutation status in pulmonary adenocarcinoma with 3D deep learning

“…As powerful algorithms of representation learning, CNNs largely reduce the necessities of hand‐craft feature engineering. Our previous study has proven the effectiveness and efficacy of deep learning in predicting the invasiveness of lung adenocarcinomas from CT images . In this regard, we addressed the problem of CT‐based EGFR mutation prediction by deep neural networks, to make our system automatic, robust and accurate.…”

Application of artificial intelligence in the diagnosis of multiple primary lung cancer

“…The core diagnosis component of the AI system was 3D DenseSharp Network, a state‐of‐the‐art multitask learning deep neural network based on 3D DenseNets, with classification and segmentation heads for diagnosing and segmenting lung nodules. In the primary study on cross‐modal pathological invasiveness prediction from CT scans, the software developer set up a single‐cohort data set, which involved 651 patients from Huadong Hospital affiliated to Fudan University, Shanghai, China . Each sample in this data set consisted of the CT scan, the corresponding pathology‐validated invasiveness label (AAH, AIS, MIA, or IA), and expert‐annotated lesion segmentation.…”

“…Based on this data set, the 3D DenseSharp Network was developed to train and predict the invasiveness labels and lesion segmentation, with a superior performance over radiologists on this task. The AI system used in this study, developed with the 3D deep learning technology, took advantage of a multicohort data set, with 10x more patients than their primary study . Using this AI system, dicom files were analyzed to screen out intrapulmonary subsolid nodules and their three‐dimensional (3D) volumes, radial lines, probability of nodules and malignancy, probable pathological patterns and other parameters.…”

“…In the primary study on cross-modal pathological invasiveness prediction from CT scans, the software developer set up a single-cohort data set, which involved 651 patients from Huadong Hospital affiliated to Fudan University, Shanghai, China. 6 Each sample in this data set consisted of the CT scan, the corresponding pathology-validated invasiveness label (AAH, AIS, MIA, or IA), and expert-annotated lesion segmentation. Based on this data set, the 3D DenseSharp Network was developed to train and predict the invasiveness labels and lesion segmentation, with a superior performance over radiologists on this task.…”

“…The AI system used in this study, developed with the 3D deep learning technology, 6,7 took advantage of a multicohort data set, with 10x more patients than their primary study. 6 Using this AI system, dicom files were analyzed to screen out intrapulmonary subsolid nodules and their threedimensional (3D) volumes, radial lines, probability of nodules and malignancy, probable pathological patterns and other parameters. (i) 3D volume was calculated by AI through pixel-level scanning and automatic segmentation of nodules.…”