Stepwise Feature Fusion: Local Guides Global

Wang, Jinfeng; Huang, Qiming; Tang, Feilong; Meng, Jia; Su, Jionglong; Song, Sifan

doi:10.1007/978-3-031-16437-8_11

Cited by 128 publications

(48 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SSFormer [3], which uses a pyramid transformer encoder and a progressive locality decoder for stepwise feature fusion with local emphasis was trained as a reference method. This state-of-the-art model efficiently segments complex morphological features of varying sizes.…”

Section: Methodsmentioning

confidence: 99%

Comparative analysis between convolutional long short-term memory networks and vision transformers for coronary calcium scoring in non-contrast CT

Shanbhag

Pieszko

Miller

et al. 2023

Medical Imaging 2023: Image Processing

View full text Add to dashboard Cite

PurposeCoronary artery calcium (CAC) scores are a well-established marker of the extent of coronary atherosclerosis. We aimed to compare state-of-the-art vision transformer for medical image segmentation with convolutional long short-term memory (convLSTM) networks for automatic CAC quantification with external validation. MethodsWe trained a convLSTM to segment CAC using 2500 non-contrast CT acquired between 2005-2012 from a single site. A SSFormer, which uses a pyramid transformer encoder and a progressive locality decoder for stepwise feature fusion with local emphasis was trained as a reference method. Both models were validated prospectively (2013-2015, n=7542) and in an external cohort (n=850). Cohen's Kappa coefficients and concordance matrices were used to assess agreement in five CAC score categories (no CAC, very low: <10, low: 10-100, moderate:101-400, and high>400). Z-test was used to assess the statistical significance of differences in kappa values. ResultsThe Cohen's kappa on the external cohort for convLSTM was 0.86 (95% Confidence Intervals [CI]: 0.84, 0.88) was significantly higher than SSFormer 0.82 (95% CI:0.8, 0.85, p=0.013). The concordance for convLSTM model was 0.81 as compared to 0.77 for the SSFormer, with a lower two-category difference for convLSTM than SSFormer (n=33/966, n=48/966) respectively. The time to derive results on a Graphics Processing Unit (111.23 milliseconds, interquartile range [IQR]:110.8, 111.7) for convLSTM as compared to SSFormer (164.4 milliseconds, IQR: 159.8, 167.1). ConclusionConvLSTM allows for significantly higher concordant CAC quantification as compared to the SSFormer and has excellent agreement with clinical CAC scores with better generalizability in the external cohort.

show abstract

Section: Methodsmentioning

confidence: 99%

Comparative analysis between convolutional long short-term memory networks and vision transformers for coronary calcium scoring in non-contrast CT

Shanbhag

Pieszko

Miller

et al. 2023

Medical Imaging 2023: Image Processing

View full text Add to dashboard Cite

show abstract

“…For example, MIA‐Net [8] uses both to capture global dependences and low‐level spatial details. Some studies [4, 6, 7, 9, 54] adopt pure Transformers for feature abstraction. For example, ColonFormer [6] and Polyp2Seg [7] rely on Transformer encoder for feature encoding and atrous convolution to further explore multi‐scale representations.…”

Section: Related Workmentioning

confidence: 99%

“…This characteristic fits well with the advantages of Transformers which do well in discovering the long‐range relations. Actually, the encoders built by Transformer [5], or Transformer encoders, have already been adopted in recent studies [4, 6–9] for robust feature extraction when doing polyp segmentation, which is also interesting to us for better performance.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Parallel matters: Efficient polyp segmentation with parallel structured feature augmentation modules

Guo

Fang

Wang

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

The large variations of polyp sizes and shapes and the close resemblances of polyps to their surroundings call for features with long-range information in rich scales and strong discrimination. This article proposes two parallel structured modules for building those features. One is the Transformer Inception module (TI) which applies Transformers with different reception fields in parallel to input features and thus enriches them with more long-range information in more scales. The other is the Local-Detail Augmentation module (LDA) which applies the spatial and channel attentions in parallel to each block and thus locally augments the features from two complementary dimensions for more object details. Integrating TI and LDA, a new Transformer encoder based framework, Parallel-Enhanced Network (PENet), is proposed, where LDA is specifically adopted twice in a coarse-tofine way for accurate prediction. PENet is efficient in segmenting polyps with different sizes and shapes without the interference from the background tissues. Experimental comparisons with state-of-the-arts methods show its merits.

show abstract

“…Zhou et al 23 also achieved improved accuracy on the BCTV dataset by replacing most of the convolutional blocks in Vnet with a 3D sliding window ViT model that implemented a 3D extension of efficient Swin-like transformer layers. 24 Not limited to the networks mentioned above, more transformerbased segmentation networks [25][26][27][28][29][30] were proposed and demonstrated state-of -the-art performance in different medical segmentation tasks.…”

Section: Introductionmentioning

confidence: 99%

Abdomen CT multi‐organ segmentation using token‐based MLP‐Mixer

et al. 2022

View full text Add to dashboard Cite

Background: Manual contouring is very labor-intensive, time-consuming, and subject to intra-and inter-observer variability. An automated deep learning approach to fast and accurate contouring and segmentation is desirable during radiotherapy treatment planning. Purpose: This work investigates an efficient deep-learning-based segmentation algorithm in abdomen computed tomography (CT) to facilitate radiation treatment planning. Methods: In this work, we propose a novel deep-learning model utilizing U-shaped multi-layer perceptron mixer (MLP-Mixer) and convolutional neural network (CNN) for multi-organ segmentation in abdomen CT images. The proposed model has a similar structure to V-net, while a proposed MLP-Convolutional block replaces each convolutional block. The MLP-Convolutional block consists of three components: an early convolutional block for local features extraction and feature resampling, a token-based MLP-Mixer layer for capturing global features with high efficiency, and a token projector for pixel-level detail recovery. We evaluate our proposed network using: (1) an institutional dataset with 60 patient cases and (2) a public dataset (BCTV) with 30 patient cases. The network performance was quantitatively evaluated in three domains: (1) volume similarity between the ground truth contours and the network predictions using the Dice score coefficient (DSC), sensitivity, and precision; (2) surface similarity using Hausdorff distance (HD), mean surface distance (MSD) and residual mean square distance (RMS); and (3) the computational complexity reported by the number of network parameters, training time, and inference time. The performance of the proposed network is compared with other state-of -the-art networks. Results: In the institutional dataset, the proposed network achieved the following volume similarity measures when averaged over all organs: DSC = 0.912, sensitivity = 0.917, precision = 0.917, average surface similarities were HD = 11.95 mm, MSD = 1.90 mm, RMS = 3.86 mm. The proposed network achieved DSC = 0.786 and HD = 9.04 mm on the public dataset. The network also shows statistically significant improvement, which is evaluated by a two-tailed Wilcoxon Mann-Whitney U test, on right lung (MSD where the maximum p-value is 0.001), spinal cord (sensitivity, precision, HD, RMSD where p-value ranges from 0.001 to 0.039), and stomach (DSC where the maximum p-value is 0.01) over all other competing networks. On the public dataset, the network report statistically significant improvement, which is shown by the Wilcoxon Mann-Whitney test, on pancreas (HD where the maximum p-value is 0.006), left (HD where

show abstract

Stepwise Feature Fusion: Local Guides Global

Cited by 128 publications

References 18 publications

Comparative analysis between convolutional long short-term memory networks and vision transformers for coronary calcium scoring in non-contrast CT

Comparative analysis between convolutional long short-term memory networks and vision transformers for coronary calcium scoring in non-contrast CT

Parallel matters: Efficient polyp segmentation with parallel structured feature augmentation modules

Abdomen CT multi‐organ segmentation using token‐based MLP‐Mixer

Contact Info

Product

Resources

About