LCDEiT: A Linear Complexity Data-Efficient Image Transformer for MRI Brain Tumor Classification

Abstract

Current deep learning-assisted brain tumor classification models sustain inductive bias and parameter dependency problems for extracting texture-based image information. Thereby concerning these problems, the recent development of the vision transformer model has substituted the DL model for classification tasks. However, the high performance of the vision transformer model depends on a large-scale dataset as well as self-attention calculations between the number of image patches which result in a quadratic computational complexity. To address these problems, the vision transformer must be data-efficient to be well-trained with a limited amount of data, and the computational complexity must be linear with the number of image patches. Consequently, this paper presents a novel linear-complexity data-efficient image transformer called LCDEiT for training with small-size datasets by using a teacher-student strategy and linear computational complexity concerning the number of patches using an external attention mechanism. The teacher model comprised a custom gated-pooled convolutional neural network to provide knowledge to the transformer-based student model for the classification of MRI brain tumors. The average classification accuracy and F1-score for two benchmark datasets including Figshare and BraTS-21 are found 98.11% and 97.86% and 93.69% and 93.68% respectively. The results indicate that the proposed model could have a great impact on medical imaging-based diagnosis where data availability and faster computations are the main concern.

Cardiac Healthcare Digital Twins Supported by Artificial Intelligence-Based Algorithms and Extended Reality—A Systematic Review

“…Originally, they were networks used for natural language processing (NLP). Their effectiveness in these tasks resulted in the development of transformers such as the Detection Transformer (DETR) for tasks related to vision analysis [193], the Swin-Transformer [61], the Vision Transformer (ViT) [194], and the Data-Efficient Image Transformer (DeiT) [194]. The DETR is dedicated to object detection which also includes manual analytical processes, and it uses CNN to learn 2D representations of the input data (images).…”

Understanding the brain with attention: A survey of transformers in brain sciences

“…The methods described above are often applied in the diagnosis of brain tumors and AD and have generally improved diagnostic efficiency through the remarkable performance of Transformers. In these models, the Transformer encoder often implements modular integration, [77,82,84] which not only retains the complete function to stabilize its performance but also obtains different parameter matrices according to specific tasks, improving the expressiveness of downstream tasks. Some dual-branch designs retain CNNs and Transformers to build local and global feature extraction, [79,80] making up for the lack of single feature extraction.…”

“…They crossed vectors to calculate attention weights in different attention heads, flexibly learning more important pathological features. Considering that Transformers rely on large-scale datasets for self-attention calculation, Ferdous et al [82] proposed a Linear-Complexity Data-Efficient image Transformer (LCDEiT), which used the teacher-student strategy and external attention mechanism to gain low-complexity calculation, leading to rapid brain tumor classification. Sarasua et al [83] introduced a spatio-temporal network for 3D anatomical Meshes (TransforMesh) that exploited Transformers to incorporate heterogeneous trajectories, which led to the prediction of neuroanatomical changes.…”

“…They crossed vectors to calculate attention weights in different attention heads, flexibly learning more important pathological features. Considering that Transformers rely on large‐scale datasets for self‐attention calculation, Ferdous et al [82] . proposed a Linear‐Complexity Data‐Efficient image Transformer (LCDEiT), which used the teacher‐student strategy and external attention mechanism to gain low‐complexity calculation, leading to rapid brain tumor classification.…”

“…Aloraini et al [79] BECNN [79] 2023 Tumor classification [80] ASI-DBNet [80] 2023 Tumor progression Microscopic image 2500 CE AdamW 95.24 -Anaya-Isaza et al [81] Crosstransformer [81] 2023 Ferdous et al [82] LCDEiT [82] 2023 [83] TransforMesh [83] 2021 Detection of neuroanatomical changes in AD T1/T2*/FLAIR -MAE with regularization constant ---Samak et al [84] TransSOP [84] 2023 mRS score NCCT 500 -Adam 80.00 59.00…”