Squeeze-and-Excitation Networks

Hu, J. F.; Shen, Li; Albanie, Samuel; Sun, Gang; Wu, Enhua

doi:10.48550/arxiv.1709.01507

Cited by 453 publications

(628 citation statements)

References 0 publications

Supporting

Mentioning

620

Contrasting

Unclassified

Order By: Relevance

“…Note, some papers have also focused on ResNet-50 training [2,27,49], but they have either modified the architecture or changed the resolution, which does not allow for a direct comparison to the original ResNet-50 at resolution 224×224. For instance, Lee et al [27] use ResNet-D [14] with SE attention [20]. Bello et al [2] also optimize ResNet without architectural changes, but they don't report competitive results for ResNet-50 at 224×224.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

ResNet strikes back: An improved training procedure in timm

Wightman¹,

Touvron²

2021

Preprint

112

View full text Add to dashboard Cite

The influential Residual Networks designed by He et al. remain the gold-standard architecture in numerous scientific publications. They typically serve as the default architecture in studies, or as baselines when new architectures are proposed. Yet there has been significant progress on best practices for training neural networks since the inception of the ResNet architecture in 2015. Novel optimization & dataaugmentation have increased the effectiveness of the training recipes.In this paper, we re-evaluate the performance of the vanilla ResNet-50 when trained with a procedure that integrates such advances. We share competitive training settings and pre-trained models in the timm open-source library, with the hope that they will serve as better baselines for future work. For instance, with our more demanding training setting, a vanilla ResNet-50 reaches 80.4% top-1 accuracy at resolution 224×224 on ImageNet-val without extra data or distillation. We also report the performance achieved with popular models with our training procedure.

show abstract

Section: Related Workmentioning

confidence: 99%

“…We solely consider the training recipe. Therefore we exclude all variations of the ResNet-50 such as SE-ResNet-50 [20] or ResNet-50-D [14], which usually improve the accuracy under the same training procedure. In summary, in this paper,…”

Section: Introductionmentioning

confidence: 99%

ResNet strikes back: An improved training procedure in timm

Wightman¹,

Touvron²

2021

Preprint

112

View full text Add to dashboard Cite

show abstract

“…Besides, as can be indicated from the bowl shape of curves, a medium number of features in each channel performs best when F × C is fixed. The reason is that the number of channels is too small and there is not enough degrees of freedom for dynamic adjustment of attention maps when F is too large, while each channel does not contain enough features to effectively capture the global information [40] when F is too small. So, we choose to reshape the feature vector into 192 channels with 16 features in each channel.…”

Section: A Impacts Of Network Parametersmentioning

confidence: 99%

“…• As is illustrated in the third subfigure, all scale factors in the fourth attention map are 0.5, which is due to the zero output of the former ReLU activation function and the Sigmoid activation function used to predict the attention map. Therefore, the last attention module is actually useless and can be removed during testing to further reduce the complexity [40].…”

Section: The Role Of Attentionmentioning

confidence: 99%

See 1 more Smart Citation

An Attention-Aided Deep Learning Framework for Massive MIMO Channel Estimation

Gao¹,

Hu²,

Zhong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Channel estimation is one of the key issues in practical massive multiple-input multiple-output (MIMO) systems. Compared with conventional estimation algorithms, deep learning (DL) based ones have exhibited great potential in terms of performance and complexity. In this paper, an attention mechanism, exploiting the channel distribution characteristics, is proposed to improve the estimation accuracy of highly separable channels with narrow angular spread by realizing the "divide-and-conquer" policy. Specifically, we introduce a novel attention-aided DL channel estimation framework for conventional massive MIMO systems and devise an embedding method to effectively integrate the attention mechanism into the fully connected neural network for the hybrid analog-digital (HAD) architecture. Simulation results show that in both scenarios, the channel estimation performance is significantly improved with the aid of attention at the cost of small complexity overhead. Furthermore, strong robustness under different system and channel parameters can be achieved by the proposed approach, which further strengthens its practical value. We also investigate the distributions of learned attention maps to reveal the role of attention, which endows the proposed approach with a certain degree of interpretability.

show abstract

Error detection using a multi‐channel hybrid network with a low‐resolution detector in patient‐specific quality assurance

Yan,

Shi,

Xue

et al. 2024

J Applied Clin Med Phys

View full text Add to dashboard Cite

PurposeThis study aimed to develop a hybrid multi‐channel network to detect multileaf collimator (MLC) positional errors using dose difference (DD) maps and gamma maps generated from low‐resolution detectors in patient‐specific quality assurance (QA) for Intensity Modulated Radiation Therapy (IMRT).MethodsA total of 68 plans with 358 beams of IMRT were included in this study. The MLC leaf positions of all control points in the original IMRT plans were modified to simulate four types of errors: shift error, opening error, closing error, and random error. These modified plans were imported into the treatment planning system (TPS) to calculate the predicted dose, while the PTW seven29 phantom was utilized to obtain the measured dose distributions. Based on the measured and predicted dose, DD maps and gamma maps, both with and without errors, were generated, resulting in a dataset with 3222 samples. The network's performance was evaluated using various metrics, including accuracy, sensitivity, specificity, precision, F1‐score, ROC curves, and normalized confusion matrix. Besides, other baseline methods, such as single‐channel hybrid network, ResNet‐18, and Swin‐Transformer, were also evaluated as a comparison.ResultsThe experimental results showed that the multi‐channel hybrid network outperformed other methods, demonstrating higher average precision, accuracy, sensitivity, specificity, and F1‐scores, with values of 0.87, 0.89, 0.85, 0.97, and 0.85, respectively. The multi‐channel hybrid network also achieved higher AUC values in the random errors (0.964) and the error‐free (0.946) categories. Although the average accuracy of the multi‐channel hybrid network was only marginally better than that of ResNet‐18 and Swin Transformer, it significantly outperformed them regarding precision in the error‐free category.ConclusionThe proposed multi‐channel hybrid network exhibits a high level of accuracy in identifying MLC errors using low‐resolution detectors. The method offers an effective and reliable solution for promoting quality and safety of IMRT QA.

show abstract

Squeeze-and-Excitation Networks

Cited by 453 publications

References 0 publications

ResNet strikes back: An improved training procedure in timm

ResNet strikes back: An improved training procedure in timm

An Attention-Aided Deep Learning Framework for Massive MIMO Channel Estimation

Error detection using a multi‐channel hybrid network with a low‐resolution detector in patient‐specific quality assurance

Contact Info

Product

Resources

About