Combining CNNs with Transformer for Multimodal 3D MRI Brain Tumor Segmentation

Dobko, Mariia; Kolinko, Danylo-Ivan; Viniavskyi, Ostap; Yelisieiev, Yurii

doi:10.1007/978-3-031-09002-8_21

Cited by 5 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, the spatial attention mechanism allowed the model to extract the important regions of the image adaptively. Different from the disadvantage of the poor data correlation caused by dividing the feature map, the feature map could be directly converted into tokens by spatial pooling without segmentation, which effectively ensured the consistency of feature information [ 80 , 81 , 82 ]. In this paper, six groups of spatial attention mechanisms were set for each Transformer module to extract the network, i.e., the number of tokens input to each Transformer module was four.…”

Section: Methodsmentioning

confidence: 99%

Table Tennis Track Detection Based on Temporal Feature Multiplexing Network

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

Recording the trajectory of table tennis balls in real-time enables the analysis of the opponent’s attacking characteristics and weaknesses. The current analysis of the ball paths mainly relied on human viewing, which lacked certain theoretical data support. In order to solve the problem of the lack of objective data analysis in the research of table tennis competition, a target detection algorithm-based table tennis trajectory extraction network was proposed to record the trajectory of the table tennis movement in video. The network improved the feature reuse rate in order to achieve a lightweight network and enhance the detection accuracy. The core of the network was the “feature store & return” module, which could store the output of the current network layer and pass the features to the input of the network layer at the next moment to achieve efficient reuse of the features. In this module, the Transformer model was used to secondarily process the features, build the global association information, and enhance the feature richness of the feature map. According to the designed experiments, the detection accuracy of the network was 96.8% for table tennis and 89.1% for target localization. Moreover, the parameter size of the model was only 7.68 MB, and the detection frame rate could reach 634.19 FPS using the hardware for the tests. In summary, the network designed in this paper has the characteristics of both lightweight and high precision in table tennis detection, and the performance of the proposed model significantly outperforms that of the existing models.

show abstract

Section: Methodsmentioning

confidence: 99%

Table Tennis Track Detection Based on Temporal Feature Multiplexing Network

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The recent success of transformer architecture in vision tasks ( 11 , 12 ) has shown benefits in learning global contextual information. New network designs with vision transformers have emerged for medical image segmentation ( 13 , 14 ) and achieved state-of-the-art (SOTA) performance in brain tumor segmentation ( 15 – 17 ). However, the supervised training of vision transformers typically requires a large amount of densely annotated images, otherwise there is a high risk of overfitting.…”

Section: Introductionmentioning

confidence: 99%

Improving brain tumor segmentation with anatomical prior-informed pre-training

Wang,

Li,

Wang

et al. 2023

Front. Med.

View full text Add to dashboard Cite

IntroductionPrecise delineation of glioblastoma in multi-parameter magnetic resonance images is pivotal for neurosurgery and subsequent treatment monitoring. Transformer models have shown promise in brain tumor segmentation, but their efficacy heavily depends on a substantial amount of annotated data. To address the scarcity of annotated data and improve model robustness, self-supervised learning methods using masked autoencoders have been devised. Nevertheless, these methods have not incorporated the anatomical priors of brain structures.MethodsThis study proposed an anatomical prior-informed masking strategy to enhance the pre-training of masked autoencoders, which combines data-driven reconstruction with anatomical knowledge. We investigate the likelihood of tumor presence in various brain structures, and this information is then utilized to guide the masking procedure.ResultsCompared with random masking, our method enables the pre-training to concentrate on regions that are more pertinent to downstream segmentation. Experiments conducted on the BraTS21 dataset demonstrate that our proposed method surpasses the performance of state-of-the-art self-supervised learning techniques. It enhances brain tumor segmentation in terms of both accuracy and data efficiency.DiscussionTailored mechanisms designed to extract valuable information from extensive data could enhance computational efficiency and performance, resulting in increased precision. It's still promising to integrate anatomical priors and vision approaches.

show abstract