Deep 3D Neural Network for Brain Structures Segmentation Using Self-Attention Modules in MRI Images

Abstract

In recent years, the use of deep learning-based models for developing advanced healthcare systems has been growing due to the results they can achieve. However, the majority of the proposed deep learning-models largely use convolutional and pooling operations, causing a loss in valuable data and focusing on local information. In this paper, we propose a deep learning-based approach that uses global and local features which are of importance in the medical image segmentation process. In order to train the architecture, we used extracted three-dimensional (3D) blocks from the full magnetic resonance image resolution, which were sent through a set of successive convolutional neural network (CNN) layers free of pooling operations to extract local information. Later, we sent the resulting feature maps to successive layers of self-attention modules to obtain the global context, whose output was later dispatched to the decoder pipeline composed mostly of upsampling layers. The model was trained using the Mindboggle-101 dataset. The experimental results showed that the self-attention modules allow segmentation with a higher Mean Dice Score of 0.90 ± 0.036 compared with other UNet-based approaches. The average segmentation time was approximately 0.038 s per brain structure. The proposed model allows tackling the brain structure segmentation task properly. Exploiting the global context that the self-attention modules incorporate allows for more precise and faster segmentation. We segmented 37 brain structures and, to the best of our knowledge, it is the largest number of structures under a 3D approach using attention mechanisms.

Is Attention all You Need in Medical Image Analysis? A Review

“…Following further validation using CT data from other organs, this hybrid approach can potentially have applicability in terms of detecting small and irregular lesions across different diseases and organ areas. Other studies focused on segmentation of large-scale MRI data [58][59][60][61][62].…”

Empowering Vision Transformer by Optimal Network Hyper-Parameter Selection for Whole Pelvis Prostate Planning Target Volume Auto-Segmentation

“…Self-attention is a crucial component of the transformer, enabling the representation of the degree of impact as a correlation by shifting a single sequence to different sequences, thus handling the global receptive field intrinsically [23][24][25][26]. Furthermore, instead of updating the convolution filters as typically done in a CNN [27], the self-attention mechanism updates three matrices in parallel, namely query (Q), key (K), and value (V) vectors.…”

GPU-Based Parallel Processing Techniques for Enhanced Brain Magnetic Resonance Imaging Analysis: A Review of Recent Advances

“…Also, a highly appropriate SIMD algorithm that operates on thousands of threads concurrently is executed by today's GPUs [9]. On the other hand, the most frequently used medical imaging modality for brain imaging is magnetic resonance imaging (MRI), followed by computed tomography (CT), positron emission tomography (PET), and ultrasound [10][11][12][13]. In basic terms, MRI has been widely utilized to analyze the anatomy of the entire brain [14].…”