GMILT: A Novel Transformer Network That Can Noninvasively Predict EGFR Mutation Status

Zhao, Wei; Chen, Weidao; Li, Ge; Du, Lan; Yang, Jiancheng; Chen, Yanjing; Jiang, Yingjia; Wu, Jiangfen; Ni, Bingbing; Sun, Yanjun; Wang, Shaokang; Sun, Yueming; Li, Ming; Liu, Jun

doi:10.1109/tnnls.2022.3190671

Cited by 11 publications

(5 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach is also robust against human variance, which is high, as shown in our inter-rater consistency analysis. Finally, the current method focuses on modeling only the CT modality, whereas ideally, clinicians use a variety of information, such as smoking history and multiomics information (52,53) to better estimate the metastasis and malignancy of solid PNs. Aggregating such information in our modeling may further boost its diagnostic performance.…”

Section: Discussionmentioning

confidence: 99%

Deep learning predicts malignancy and metastasis of solid pulmonary nodules from CT scans

Kuang

et al. 2023

Front. Med.

Self Cite

View full text Add to dashboard Cite

In the clinic, it is difficult to distinguish the malignancy and aggressiveness of solid pulmonary nodules (PNs). Incorrect assessments may lead to delayed diagnosis and an increased risk of complications. We developed and validated a deep learning-based model for the prediction of malignancy as well as local or distant metastasis in solid PNs based on CT images of primary lesions during initial diagnosis. In this study, we reviewed the data from multiple patients with solid PNs at our institution from 1 January 2019 to 30 April 2022. The patients were divided into three groups: benign, Ia-stage lung cancer, and T1-stage lung cancer with metastasis. Each cohort was further split into training and testing groups. The deep learning system predicted the malignancy and metastasis status of solid PNs based on CT images, and then we compared the malignancy prediction results among four different levels of clinicians. Experiments confirmed that human–computer collaboration can further enhance diagnostic accuracy. We made a held-out testing set of 134 cases, with 689 cases in total. Our convolutional neural network model reached an area under the ROC (AUC) of 80.37% for malignancy prediction and an AUC of 86.44% for metastasis prediction. In observer studies involving four clinicians, the proposed deep learning method outperformed a junior respiratory clinician and a 5-year respiratory clinician by considerable margins; it was on par with a senior respiratory clinician and was only slightly inferior to a senior radiologist. Our human–computer collaboration experiment showed that by simply adding binary human diagnosis into model prediction probabilities, model AUC scores improved to 81.80–88.70% when combined with three out of four clinicians. In summary, the deep learning method can accurately diagnose the malignancy of solid PNs, improve its performance when collaborating with human experts, predict local or distant metastasis in patients with T1-stage lung cancer, and facilitate the application of precision medicine.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep learning predicts malignancy and metastasis of solid pulmonary nodules from CT scans

Kuang

et al. 2023

Front. Med.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Specifically, due to the inherent glioma nature of tumor heterogeneity and lesion boundary diffusion, quite a lot of noise mixed with information related to the IDH genotyping. Compared with CNN, the Transformer network was more prudent to the signal noise [ 37 , 38 , 39 ]. (ii) Hierarchical architecture spurred by the translation invariance advantage of CNN had the flexibility to model at various scales.…”

Section: Discussionmentioning

confidence: 99%

Swin Transformer Improves the IDH Mutation Status Prediction of Gliomas Free of MRI-Based Tumor Segmentation

Shen

et al. 2022

JCM

Self Cite

View full text Add to dashboard Cite

Background: Deep learning (DL) could predict isocitrate dehydrogenase (IDH) mutation status from MRIs. Yet, previous work focused on CNNs with refined tumor segmentation. To bridge the gap, this study aimed to evaluate the feasibility of developing a Transformer-based network to predict the IDH mutation status free of refined tumor segmentation. Methods: A total of 493 glioma patients were recruited from two independent institutions for model development (TCIA; N = 259) and external test (AHXZ; N = 234). IDH mutation status was predicted directly from T2 images with a Swin Transformer and conventional ResNet. Furthermore, to investigate the necessity of refined tumor segmentation, seven strategies for the model input image were explored: (i) whole tumor slice; (ii–iii) tumor mask and/or not edema; (iv–vii) tumor bounding box of 0.8, 1.0, 1.2, 1.5 times. Performance comparison was made among the networks of different architectures along with different image input strategies, using area under the curve (AUC) and accuracy (ACC). Finally, to further boost the performance, a hybrid model was built by incorporating the images with clinical features. Results: With the seven proposed input strategies, seven Swin Transformer models and seven ResNet models were built, respectively. Based on the seven Swin Transformer models, an averaged AUC of 0.965 (internal test) and 0.842 (external test) were achieved, outperforming 0.922 and 0.805 resulting from the seven ResNet models, respectively. When a bounding box of 1.0 times was used, Swin Transformer (AUC = 0.868, ACC = 80.7%), achieved the best results against the one that used tumor segmentation (Tumor + Edema, AUC = 0.862, ACC = 78.5%). The hybrid model that integrated age and location features into images yielded improved performance (AUC = 0.878, Accuracy = 82.0%) over the model that used images only. Conclusions: Swin Transformer outperforms the CNN-based ResNet in IDH prediction. Using bounding box input images benefits the DL networks in IDH prediction and makes the IDH prediction free of refined glioma segmentation feasible.

show abstract

“…Compared with radiomics model, the end-to-end deep learning method can extract high-order image features by using a self-learning strategy and does not require a precise annotated tumour boundary (Gong et al 2020). For a data-driven algorithm, a relatively large dataset must be collected to train the deep learning model (Wang et al 2019, Haim et al 2022, Zhao et al 2022. Previous deep learning studies have demonstrated promising performance in the risk prediction of lung cancer (Dong et al 2021, Gong et al 2021.…”

Section: Introductionmentioning

confidence: 99%

Hybrid deep multi-task learning radiomics approach for predicting EGFR mutation status of non-small cell lung cancer in CT images

Gong,

Fu,

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

Objective: Epidermal growth factor receptor (EGFR) mutation genotyping plays a pivotal role in targeted therapy for non-small cell lung cancer (NSCLC). We aimed to develop a computed tomography (CT) image-based hybrid deep radiomics model to predict EGFR mutation status in NSCLC and investigate the correlations between deep image and quantitative radiomics features.Approach: First, we retrospectively enrolled 818 patients from our centre and 131 patients from The Cancer Imaging Archive database to establish a training cohort (N = 654), an independent internal validation cohort (N = 164) and an external validation cohort (N = 131). Second, to predict EGFR mutation status, we developed three CT image-based models, namely, a multi-task deep neural network (DNN), a radiomics model and a feature fusion model. Third, we proposed a hybrid loss function to train the DNN model. Finally, to evaluate the model performance, we computed the areas under the receiver operating characteristic curves (AUCs) and decision curve analysis curves of the models.Main results: For the two validation cohorts, the feature fusion model achieved AUC values of 0.86±0.03 and 0.80±0.05, which were significantly higher than those of the single-task DNN and radiomics models (all P < 0.05). There was no significant difference between the feature fusion and the multi-task DNN models (P > 0.8). The binary prediction scores showed excellent prognostic value in predicting disease-free survival (P = 0.02) and overall survival (P < 0.005) for validation cohort 2.Significance: The results demonstrate that 1) the feature fusion and multi-task DNN models achieve significantly higher performance than that of the conventional radiomics and single-task DNN models, 2) the feature fusion model can decode the imaging phenotypes representing NSCLC heterogeneity related to both EGFR mutation and patient NSCLC prognosis, and 3) high correlations exist between some deep image and radiomics features.

show abstract

GMILT: A Novel Transformer Network That Can Noninvasively Predict EGFR Mutation Status

Cited by 11 publications

References 44 publications

Deep learning predicts malignancy and metastasis of solid pulmonary nodules from CT scans

Deep learning predicts malignancy and metastasis of solid pulmonary nodules from CT scans

Swin Transformer Improves the IDH Mutation Status Prediction of Gliomas Free of MRI-Based Tumor Segmentation

Hybrid deep multi-task learning radiomics approach for predicting EGFR mutation status of non-small cell lung cancer in CT images

Contact Info

Product

Resources

About