Conditional DETR for Fast Training Convergence

Meng, Depu; Chen, Xiaokang; Fan, Zejia; Zeng, Gang; Li, Houqiang; Yuan, Yuhui; Sun, Lei; Wang, Jingdong

doi:10.48550/arxiv.2108.06152

Cited by 14 publications

(27 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Deformable DETR (Zhu et al, 2021) directly treats 2D reference points as queries and predicts deformable sampling points for each reference point to perform deformable cross-attention operation. Conditional DETR (Meng et al, 2021) decouples the attention formulation and generates positional queries based on reference coordinates. Efficient DETR (Yao et al, 2021) introduces a dense prediction module to select top-K positions as object queries.…”

Section: Related Workmentioning

confidence: 99%

“…For example, SMCA (Gao et al, 2021) speeds up the training by applying pre-defined Gaussian maps around reference points. Conditional DETR (Meng et al, 2021) uses explicit positional embedding as positional queries for training, yielding attention maps similar to Gaussian kernels as shown in Fig. 4…”

Section: Related Workmentioning

confidence: 99%

“…Inspired by Conditional DETR (Meng et al, 2021), we concatenate the position and content information together as queries and keys in the cross-attention module, so that we can decouple the content and position contributions to the query-to-feature similarity computed as the dot product between a query and a key. To rescale the positional embeddings, we leverage the conditional spatial query (Meng et al, 2021) as well. More specifically, we learn a MLP (csq) : R D → R D to obtain a scale vector conditional on the content information and use it perform element-wise multiplication with the positional embeddings:…”

Section: Learning Anchor Boxes Directlymentioning

confidence: 99%

“…However, due to its ineffective design and use of queries, DETR suffers from significantly slow training convergence, usually requiring 500 epochs to achieve a good performance. To address this issue, many follow-up works attempted to improve the design of DETR queries for both faster training convergence and better performance (Zhu et al, 2021;Gao et al, 2021;Meng et al, 2021;. Despite all the progress, the role of the learned queries in DETR is still not fully understood or utilized.…”

Section: Introductionmentioning

confidence: 99%

“…For example, Conditional DETR learns a conditional spatial query by adapting a query based on its content feature for better matching with image features (Meng et al, 2021). Efficient DETR introduces a dense prediction module to select top-K object queries (Yao et al, 2021) and Anchor DETR formulates queries as 2D anchor points , both associating each query with a specific spatial position.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR

Liu¹,

Li²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR. This new formulation directly uses box coordinates as queries in Transformer decoders and dynamically updates them layer-by-layer. Using box coordinates not only helps using explicit positional priors to improve the queryto-feature similarity and eliminate the slow training convergence issue in DETR, but also allows us to modulate the positional attention map using the box width and height information. Such a design makes it clear that queries in DETR can be implemented as performing soft ROI pooling layer-by-layer in a cascade manner. As a result, it leads to the best performance on MS-COCO benchmark among the DETR-like detection models under the same setting, e.g., AP 45.7% using ResNet50-DC5 as backbone trained in 50 epochs. We also conducted extensive experiments to confirm our analysis and verify the effectiveness of our methods.Code is available at https://github.com/SlongLiu/DAB-DETR.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Learning Anchor Boxes Directlymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR

Liu¹,

Li²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

WCAY object detection of fractures for X-ray images of multiple sites

Chen,

Liu,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

The WCAY (weighted channel attention YOLO) model, which is meticulously crafted to identify fracture features across diverse X-ray image sites, is presented herein. This model integrates novel core operators and an innovative attention mechanism to enhance its efficacy. Initially, leveraging the benefits of dynamic snake convolution (DSConv), which is adept at capturing elongated tubular structural features, we introduce the DSC-C2f module to augment the model’s fracture detection performance by replacing a portion of C2f. Subsequently, we integrate the newly proposed weighted channel attention (WCA) mechanism into the architecture to bolster feature fusion and improve fracture detection across various sites. Comparative experiments were conducted, to evaluate the performances of several attention mechanisms. These enhancement strategies were validated through experimentation on public X-ray image datasets (FracAtlas and GRAZPEDWRI-DX). Multiple experimental comparisons substantiated the model’s efficacy, demonstrating its superior accuracy and real-time detection capabilities. According to the experimental findings, on the FracAtlas dataset, our WCAY model exhibits a notable 8.8% improvement in mean average precision (mAP) over the original model. On the GRAZPEDWRI-DX dataset, the mAP reaches 64.4%, with a detection accuracy of 93.9% for the “fracture” category alone. The proposed model represents a substantial improvement over the original algorithm compared to other state-of-the-art object detection models. The code is publicly available at https://github.com/cccp421/Fracture-Detection-WCAY . Supplementary Information The online version contains supplementary material available at 10.1038/s41598-024-77878-6.

show abstract

DN-DETR: Accelerate DETR Training by Introducing Query DeNoising

Li,

Zhang,

Liu

et al. 2024

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

We present in this paper a novel denoising training method to speed up DETR (DEtection TRansformer) training and offer a deepened understanding of the slow convergence issue of DETR-like methods. We show that the slow convergence results from the instability of bipartite graph matching which causes inconsistent optimization goals in early training stages. To address this issue, except for the Hungarian loss, our method additionally feeds GT bounding boxes with noises into the Transformer decoder and trains the model to reconstruct the original boxes, which effectively reduces the bipartite graph matching difficulty and leads to faster convergence. Our method is universal and can be easily plugged into any DETR-like method by adding dozens of lines of code to achieve a remarkable improvement. As a result, our DN-DETR results in a remarkable improvement (+1.9AP) under the same setting and achieves 46.0 AP and 49.5 AP trained for 12 and 50 epochs with the ResNet-50 backbone. Compared with the baseline under the same setting, DN-DETR achieves comparable performance with 50% training epochs. We also demonstrate the effectiveness of denoising training in CNN-based detectors (Faster R-CNN), segmentation models (Mask2Former, Mask DINO), and more DETR-based models (DETR, Anchor DETR, Deformable DETR). Code is available at https://github.com/IDEA-Research/DN-DETR.

show abstract

Conditional DETR for Fast Training Convergence

Cited by 14 publications

References 0 publications

DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR

DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR

WCAY object detection of fractures for X-ray images of multiple sites

DN-DETR: Accelerate DETR Training by Introducing Query DeNoising

Contact Info

Product

Resources

About