P2T: Pyramid Pooling Transformer for Scene Understanding

Wu, Yu-Huan; Liu, Yun; Zhan, Xin; Cheng, Ming–Ming

doi:10.1109/tpami.2022.3202765

Cited by 132 publications

(50 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…VGG16 [31], ResNet50 [32], and MobileNetv2 [50]) and Transformer backbones (e.g. Swin Transformer [24] and P2T [38]) to evaluate the effectiveness of our proposed asymmetric architecture. From the quantitative comparison results in Table 2, we can see that both symmetric and asymmetric two‐stream structures, Tranformer‐based methods (PVTv2, Swin, and P2T) generally achieve superior performance compared to pure CNN‐based methods, which can be attributed to the powerful ability of Transformer to model long‐range dependencies.…”

Section: Methodsmentioning

confidence: 99%

“…The success of Vision Transformer [23] in the field of image recognition has led to the widespread application of Transformer models in computer vision tasks. Various Transformer backbones following the hierarchical structure of VGG [36] and ResNet [37] have emerged successively, such as Swin Transformer [24], PVT [25, 26], and P2T [38]. These Transformer‐based methods have achieved state‐of‐the‐art performance in a variety of computer vision tasks such as detection [39] and segmentation [40], demonstrating the great potential of the Transformer.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

TANet: Transformer‐based asymmetric network for RGB‐D salient object detection

Liu

Yang

Wang

et al. 2023

IET Computer Vision

View full text Add to dashboard Cite

Existing RGB‐D salient object detection methods mainly rely on a symmetric two‐stream Convolutional Neural Network (CNN)‐based network to extract RGB and depth channel features separately. However, there are two problems with the symmetric conventional network structure: first, the ability of CNN in learning global contexts is limited; second, the symmetric two‐stream structure ignores the inherent differences between modalities. In this study, a Transformer‐based asymmetric network is proposed to tackle the issues mentioned above. The authors employ the powerful feature extraction capability of Transformer to extract global semantic information from RGB data and design a lightweight CNN backbone to extract spatial structure information from depth data without pre‐training. The asymmetric hybrid encoder effectively reduces the number of parameters in the model while increasing speed without sacrificing performance. Then, a cross‐modal feature fusion module which enhances and fuses RGB and depth features with each other is designed. Finally, the authors add edge prediction as an auxiliary task and propose an edge enhancement module to generate sharper contours. Extensive experiments demonstrate that our method achieves superior performance over 14 state‐of‐the‐art RGB‐D methods on six public datasets. The code of the authors will be released at https://github.com/lc012463/TANet.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

TANet: Transformer‐based asymmetric network for RGB‐D salient object detection

Liu

Yang

Wang

et al. 2023

IET Computer Vision

View full text Add to dashboard Cite

show abstract

“…PVTv1 [39] PVTv1-L 512 × 512 44.8 -PVTv2 [40] PVTv2-B5 512 × 512 48.7 -P2T [41] P2T-L 512 × 512 49.4 -Swin-UperNet [42,47] Swin-L † 640 × 640 -53.5 FaPN-MaskFormer [10,48] Swin-L † 640 × 640 55.2 56.7 BEiT-UperNet [4,47] BEiT Since one of the main objectives of the proposed auxiliary CNN is to improve the segmentation performance in complex scenes which include small objects and require detailed local information for accurate segmentation, we show several qualitative results for the ADE20K and Cityscapes datasets. Figure 2 presents the qualitative results of the ADE20K validation dataset.…”

Section: Backbone Crop Size Miou (Ss) Miou (Ms)mentioning

confidence: 99%

“…PVT [ 39 , 40 ] borrows the pyramid structure concept in CNNs and designs a pyramid vision transformer for learning multi-scale features with high resolutions. P2T [ 41 ] implements a pooling-based self-attention module with depthwise convolutional operations for multi-scale feature learning. Ref.…”

Section: Related Workmentioning

confidence: 99%

Enhancing Mask Transformer with Auxiliary Convolution Layers for Semantic Segmentation

Xia

Kim

2023

Sensors

View full text Add to dashboard Cite

Transformer-based semantic segmentation methods have achieved excellent performance in recent years. Mask2Former is one of the well-known transformer-based methods which unifies common image segmentation into a universal model. However, it performs relatively poorly in obtaining local features and segmenting small objects due to relying heavily on transformers. To this end, we propose a simple yet effective architecture that introduces auxiliary branches to Mask2Former during training to capture dense local features on the encoder side. The obtained features help improve the performance of learning local information and segmenting small objects. Since the proposed auxiliary convolution layers are required only for training and can be removed during inference, the performance gain can be obtained without additional computation at inference. Experimental results show that our model can achieve state-of-the-art performance (57.6% mIoU) on the ADE20K and (84.8% mIoU) on the Cityscapes datasets.

show abstract

“…The original ViT [5] is a plain, non-hierarchical architecture. Various hierarchical transformers such as [17][18][19][20][21] have been presented afterwards. These methods inherit some designs from convolution based networks such as hierarchical structures, pooling and down-sampling with convolutions.…”

Section: Related Workmentioning

confidence: 99%

SegViT: Semantic Segmentation with Plain Vision Transformers

Zhang¹,

Tian²,

Tang³

et al. 2022

Preprint

View full text Add to dashboard Cite

We explore the capability of plain Vision Transformers (ViTs) for semantic segmentation and propose the SegViT. Previous ViT-based segmentation networks usually learn a pixel-level representation from the output of the ViT. Differently, we make use of the fundamental component-attention mechanism, to generate masks for semantic segmentation. Specifically, we propose the Attention-to-Mask (ATM) module, in which the similarity maps between a set of learnable class tokens and the spatial feature maps are transferred to the segmentation masks. Experiments show that our proposed SegViT using the ATM module outperforms its counterparts using the plain ViT backbone on the ADE20K dataset and achieves new state-of-the-art performance on COCO-Stuff-10K and PASCAL-Context datasets. Furthermore, to reduce the computational cost of the ViT backbone, we propose query-based down-sampling (QD) and query-based up-sampling (QU) to build a Shrunk structure. With the proposed Shrunk structure, the model can save up to 40% computations while maintaining competitive performance.

show abstract

P2T: Pyramid Pooling Transformer for Scene Understanding

Cited by 132 publications

References 58 publications

TANet: Transformer‐based asymmetric network for RGB‐D salient object detection

TANet: Transformer‐based asymmetric network for RGB‐D salient object detection

Enhancing Mask Transformer with Auxiliary Convolution Layers for Semantic Segmentation

SegViT: Semantic Segmentation with Plain Vision Transformers

Contact Info

Product

Resources

About