Grading diffuse glioma based on 2021 WHO grade using self-attention-base deep learning architecture: variable Vision Transformer (vViT)

Usuzaki, Takuma; Takahashi, Kengo; Inamori, Ryusei; Morishita, Yohei; Takagi, Hidenobu; Shizukuishi, Takashi; Toyama, Yoshitaka; Abe, Mirei; Ishikuro, Mami; Obara, Taku; Majima, Kazuhiro; Takase, Kei

doi:10.1016/j.bspc.2024.106001

Cited by 5 publications

(2 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A previous study has proposed a Vision Transformer (ViT)-inspired model, named variable ViT (vViT) that analyzes multiple sequences of different lengths [ 22 , 25 – 27 ]. The vViT simultaneously handles multimodal factors (patient characteristics, radiomic features, and MRI), calculating prediction accuracy for each factor and then integrating them into the overall performance.…”

Section: Introductionmentioning

confidence: 99%

“…The vViT simultaneously handles multimodal factors (patient characteristics, radiomic features, and MRI), calculating prediction accuracy for each factor and then integrating them into the overall performance. One strength of vViT is its ability to quantitatively evaluate, or identify, the most dominant factor by calculating the prediction accuracy for each factor in a single model [ 27 ]. This strength is attributed to that the vViT analyzes input factors separately.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Identifying key factors for predicting O6-Methylguanine-DNA methyltransferase status in adult patients with diffuse glioma: a multimodal analysis of demographics, radiomics, and MRI by variable Vision Transformer

Usuzaki,

Takahashi,

Inamori

et al. 2024

Neuroradiology

Self Cite

View full text Add to dashboard Cite

Purpose This study aimed to perform multimodal analysis by vision transformer (vViT) in predicting O6-methylguanine-DNA methyl transferase (MGMT) promoter status among adult patients with diffuse glioma using demographics (sex and age), radiomic features, and MRI. Methods The training and test datasets contained 122 patients with 1,570 images and 30 patients with 484 images, respectively. The radiomic features were extracted from enhancing tumors (ET), necrotic tumor cores (NCR), and the peritumoral edematous/infiltrated tissues (ED) using contrast-enhanced T1-weighted images (CE-T1WI) and T2-weighted images (T2WI). The vViT had 9 sectors; 1 demographic sector, 6 radiomic sectors (CE-T1WI ET, CE-T1WI NCR, CE-T1WI ED, T2WI ET, T2WI NCR, and T2WI ED), 2 image sectors (CE-T1WI, and T2WI). Accuracy and area under the curve of receiver-operating characteristics (AUC-ROC) were calculated for the test dataset. The performance of vViT was compared with AlexNet, GoogleNet, VGG16, and ResNet by McNemar and Delong test. Permutation importance (PI) analysis with the Mann–Whitney U test was performed. Results The accuracy was 0.833 (95% confidence interval [95%CI]: 0.714–0.877) and the area under the curve of receiver-operating characteristics was 0.840 (0.650–0.995) in the patient-based analysis. The vViT had higher accuracy than VGG16 and ResNet, and had higher AUC-ROC than GoogleNet (p<0.05). The ED radiomic features extracted from the T2-weighted image demonstrated the highest importance (PI=0.239, 95%CI: 0.237–0.240) among all other sectors (p<0.0001). Conclusion The vViT is a competent deep learning model in predicting MGMT status. The ED radiomic features of the T2-weighted image demonstrated the most dominant contribution.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Identifying key factors for predicting O6-Methylguanine-DNA methyltransferase status in adult patients with diffuse glioma: a multimodal analysis of demographics, radiomics, and MRI by variable Vision Transformer

Usuzaki,

Takahashi,

Inamori

et al. 2024

Neuroradiology

Self Cite

View full text Add to dashboard Cite

show abstract

Predicting EGFR Status After Radical Nephrectomy or Partial Nephrectomy for Renal Cell Carcinoma on CT Using a Self-attention-based Model: Variable Vision Transformer (vViT)

Usuzaki,

Inamori,

Ishikuro

et al. 2024

J Digit Imaging. Inform. med.

View full text Add to dashboard Cite

Objective To assess the effectiveness of the vViT model for predicting postoperative renal function decline by leveraging clinical data, medical images, and image-derived features; and to identify the most dominant factor influencing this prediction. Materials and Methods We developed two models, eGFR10 and eGFR20, to identify patients with a postoperative reduction in eGFR of more than 10 and more than 20, respectively, among renal cell carcinoma patients. The eGFR10 model was trained on 75 patients and tested on 27, while the eGFR20 model was trained on 77 patients and tested on 24. The vViT model inputs included class token, patient characteristics (age, sex, BMI), comorbidities (peripheral vascular disease, diabetes, liver disease), habits (smoking, alcohol), surgical details (ischemia time, blood loss, type and procedure of surgery, approach, operative time), radiomics, and tumor and kidney imaging. We used permutation feature importance to evaluate each sector's contribution. The performance of vViT was compared with CNN models, including VGG16, ResNet50, and DenseNet121, using McNemar and DeLong tests. Results The eGFR10 model achieved an accuracy of 0.741 and an AUC-ROC of 0.692, while the eGFR20 model attained an accuracy of 0.792 and an AUC-ROC of 0.812. The surgical and radiomics sectors were the most influential in both models. The vViT had higher accuracy and AUC-ROC than VGG16 and ResNet50, and higher AUC-ROC than DenseNet121 (p < 0.05). Specifically, the vViT did not have a statistically different AUC-ROC compared to VGG16 (p = 1.0) and ResNet50 (p = 0.7) but had a statistically different AUC-ROC compared to DenseNet121 (p = 0.87) for the eGFR10 model. For the eGFR20 model, the vViT did not have a statistically different AUC-ROC compared to VGG16 (p = 0.72), ResNet50 (p = 0.88), and DenseNet121 (p = 0.64). Conclusion The vViT model, a transformer-based approach for multimodal data, shows promise for preoperative CT-based prediction of eGFR status in patients with renal cell carcinoma.

show abstract

Predicting isocitrate dehydrogenase status among adult patients with diffuse glioma using patient characteristics, radiomic features, and magnetic resonance imaging: Multi-modal analysis by variable vision transformer

Usuzaki,

Inamori,

Shizukuishi

et al. 2024

Magnetic Resonance Imaging

View full text Add to dashboard Cite

Grading diffuse glioma based on 2021 WHO grade using self-attention-base deep learning architecture: variable Vision Transformer (vViT)

Cited by 5 publications

References 35 publications

Identifying key factors for predicting O6-Methylguanine-DNA methyltransferase status in adult patients with diffuse glioma: a multimodal analysis of demographics, radiomics, and MRI by variable Vision Transformer

Identifying key factors for predicting O6-Methylguanine-DNA methyltransferase status in adult patients with diffuse glioma: a multimodal analysis of demographics, radiomics, and MRI by variable Vision Transformer

Predicting EGFR Status After Radical Nephrectomy or Partial Nephrectomy for Renal Cell Carcinoma on CT Using a Self-attention-based Model: Variable Vision Transformer (vViT)

Predicting isocitrate dehydrogenase status among adult patients with diffuse glioma using patient characteristics, radiomic features, and magnetic resonance imaging: Multi-modal analysis by variable vision transformer

Contact Info

Product

Resources

About