Classification of Brain Tumor from Magnetic Resonance Imaging Using Vision Transformers Ensembling

Tummala, Sudhakar; Kadry, Seifedine; Bukhari, Syed Ahmad Chan; Rauf, Hafiz Tayyab

doi:10.3390/curroncol29100590

Cited by 104 publications

(40 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The lower accuracy of ConvNext tiny could be attributed to the fact that the benefit of the depth of the neural network of the tiny architectures of ConvNext tiny and its depthwise convolution are often not observed until the higher-order data of greater magnitude are used for training. Previous studies have demonstrated that the classification accuracy of the VIT-base is equivalent to Resnet only when the size of the images is scaled to 384 × 384, and the convolutional neural network outperforms VIT-base when the size of the images is 224 × 224 [ 26 ]. The number of parameters and flops in ResNet34 is substantially lower than those of the VIT-base and Convnext tiny, which allow significant savings in computational resources.…”

Section: Discussionmentioning

confidence: 99%

Noninvasive Classification of Glioma Subtypes Using Multiparametric MRI to Improve Deep Learning

et al. 2022

View full text Add to dashboard Cite

Background: Deep learning (DL) methods can noninvasively predict glioma subtypes; however, there is no set paradigm for the selection of network structures and input data, including the image combination method, image processing strategy, type of numeric data, and others. Purpose: To compare different combinations of DL frameworks (ResNet, ConvNext, and vision transformer (VIT)), image preprocessing strategies, magnetic resonance imaging (MRI) sequences, and numerical data for increasing the accuracy of DL models for differentiating glioma subtypes prior to surgery. Methods: Our dataset consisted of 211 patients with newly diagnosed gliomas who underwent preoperative MRI with standard and diffusion-weighted imaging methods. Different data combinations were used as input for the three different DL classifiers. Results: The accuracy of the image preprocessing strategies, including skull stripping, segment addition, and individual treatment of slices, was 5%, 10%, and 12.5% higher, respectively, than that of the other strategies. The accuracy increased by 7.5% and 10% following the addition of ADC and numeric data, respectively. ResNet34 exhibited the best performance, which was 5% and 17.5% higher than that of ConvNext tiny and VIT-base, respectively. Data Conclusions: The findings demonstrated that the addition of quantitatively numeric data, ADC images, and effective image preprocessing strategies improved model accuracy for datasets of similar size. The performance of ResNet was superior for small or medium datasets.

show abstract

Section: Discussionmentioning

confidence: 99%

Noninvasive Classification of Glioma Subtypes Using Multiparametric MRI to Improve Deep Learning

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The detailed computational process for the Transformer module applied in computer vision is shown below. The calculation process of the Transformer model differed from the structure of the model used in natural language processing, mainly in that the data accepted by the Transformer model was one-dimensional vector, so the use of Transformer in the field of computational vision needed to perform preprocessing operations [ 48 , 49 , 50 , 51 ]. Firstly, the feature map with parameter size

was equally divided into a sequence of feature blocks

, where p was the length and width of the feature blocks and N was the number of feature blocks partitioned from the feature map.…”

Section: Related Workmentioning

confidence: 99%

Table Tennis Track Detection Based on Temporal Feature Multiplexing Network

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

Recording the trajectory of table tennis balls in real-time enables the analysis of the opponent’s attacking characteristics and weaknesses. The current analysis of the ball paths mainly relied on human viewing, which lacked certain theoretical data support. In order to solve the problem of the lack of objective data analysis in the research of table tennis competition, a target detection algorithm-based table tennis trajectory extraction network was proposed to record the trajectory of the table tennis movement in video. The network improved the feature reuse rate in order to achieve a lightweight network and enhance the detection accuracy. The core of the network was the “feature store & return” module, which could store the output of the current network layer and pass the features to the input of the network layer at the next moment to achieve efficient reuse of the features. In this module, the Transformer model was used to secondarily process the features, build the global association information, and enhance the feature richness of the feature map. According to the designed experiments, the detection accuracy of the network was 96.8% for table tennis and 89.1% for target localization. Moreover, the parameter size of the model was only 7.68 MB, and the detection frame rate could reach 634.19 FPS using the hardware for the tests. In summary, the network designed in this paper has the characteristics of both lightweight and high precision in table tennis detection, and the performance of the proposed model significantly outperforms that of the existing models.

show abstract

“…Tummala et al. evaluated a standard ViT model ensemble's ability to diagnose BTs from T1‐weighted (T1w) MRI [31] and revealed the better performance of the ensemble model compared to CNN models for classifying BTs from MRI data, with overall accuracy and specificity of 98.7% and 99.4%, respectively. The test classification accuracy for GBM with the same ensemble model was 100%.…”

Section: The Roles Of Transformers In Mri‐ and Histopathology‐based B...mentioning

confidence: 99%

“…More work should be done to explore the potential connection between the transformer structure and the convolutional network structure to improve the performance of the segmentation network Tummala et al [31] ViT MRI Classification/grading of brain cancer…”

Section: Not Applicablementioning

confidence: 99%

“…The vision transformer (ViT), the first work to directly apply Transformer architecture on nonoverlapping image patches for the image classification task [66] and the transformer's adapted version for image analysis, is stated to work better than CNN models for large data. [67] Tummala et al evaluated a standard ViT model ensemble's ability to diagnose BTs from T1-weighted (T1w) MRI [31] and revealed the better performance of the ensemble model compared to CNN models for classifying BTs from MRI data, with overall accuracy and specificity of 98.7% and 99.4%, respectively. The test classification accuracy for GBM with the same ensemble model was 100%.…”

Section: The Roles Of Transformers In Mri-and Histopathology-based Br...mentioning

confidence: 99%

See 1 more Smart Citation

Potential roles of transformers in brain tumor diagnosis and treatment

Lan

Zou

Qin

et al. 2023

Brain-X

View full text Add to dashboard Cite

Brain tumor (BT) is one of many malignancies that have substantially enhanced global human morbidity and mortality rates. Early detection and characterization of glioma are essential for effective preventive strategies. Currently, the use of Transformers, a deep learning model for BT diagnosis and treatment, is attracting significant attention. The transformer self‐attention mechanism automatically learns the associations between input data for efficient processing and analysis. Research indicates that Transformers could play an essential role in the BT segmentation of magnetic resonance imaging (MRI) images, the MRI and histopathology‐based grading of brain cancer, BT molecular expression prediction, the classification of primary brain metastasis sites, voxel‐level dose and BT radiotherapy outcome prediction, synergistic prediction, and the pathway deconvolution of drug combinations. In this review, the feasibility, accuracy, and applicability of various algorithms are systematically analyzed and their prospects are discussed. Overall, this review aimed to discuss and provide an overview of the increasing applications of Transformers in real‐time BT detection and therapy, indicating their broad prospects and potential. In the future, Transformers are expected to be increasingly used for the diagnosis and subsequent treatment of BT because of the continuous development and improvement of Transformer‐based deep learning technology. However, more work is required to investigate their properties for anomaly detection, medical image classification, network design development, and application to other medical data.

show abstract

Classification of Brain Tumor from Magnetic Resonance Imaging Using Vision Transformers Ensembling

Cited by 104 publications

References 50 publications

Noninvasive Classification of Glioma Subtypes Using Multiparametric MRI to Improve Deep Learning

Noninvasive Classification of Glioma Subtypes Using Multiparametric MRI to Improve Deep Learning

Table Tennis Track Detection Based on Temporal Feature Multiplexing Network

Potential roles of transformers in brain tumor diagnosis and treatment

Contact Info

Product

Resources

About