Tube-Embedded Transformer for Pixel Prediction

Zhang, Xiaoya; Zhang, Shumin; Cui, Zhen; Li, Zechao; Xie, Jin; Yang, Jian

doi:10.1109/tmm.2022.3147664

Cited by 9 publications

(4 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[53] a spatial information-guided convolution is proposed to enhance the spatial adaptability, which incorporates geometric information into the feature learning process by generating spatially adaptive convolutional weights. Zhang et al [71] propose a coarse-to-fine decoder to generate the feature based on the transformer mechanism, which achieves pixel-level multi-task prediction.…”

Section: B Rgb-d Semantic Segmentationmentioning

confidence: 99%

Pixel Difference Convolutional Network for RGB-D Semantic Segmentation

Yang¹,

Bai²,

Sun³

et al. 2023

Preprint

View full text Add to dashboard Cite

RGB-D semantic segmentation can be advanced with convolutional neural networks due to the availability of Depth data. Although objects cannot be easily discriminated by just the 2D appearance, with the local pixel difference and geometric patterns in Depth, they can be well separated in some cases. Considering the fixed grid kernel structure, CNNs are limited to lack the ability to capture detailed, finegrained information and thus cannot achieve accurate pixellevel semantic segmentation. To solve this problem, we propose a Pixel Difference Convolutional Network (PDCNet) to capture detailed intrinsic patterns by aggregating both intensity and gradient information in the local range for Depth data and global range for RGB data, respectively. Precisely, PDCNet consists of a Depth branch and an RGB branch. For the Depth branch, we propose a Pixel Difference Convolution (PDC) to consider local and detailed geometric information in Depth data via aggregating both intensity and gradient information. For the RGB branch, we contribute a lightweight Cascade Large Kernel (CLK) to extend PDC, namely CPDC, to enjoy global contexts for RGB data and further boost performance. Consequently, both modal data's local and global pixel differences are seamlessly incorporated into PDCNet during the information propagation process. Experiments on two challenging benchmark datasets, i.e., NYUDv2 and SUN RGB-D reveal that our PDCNet achieves state-of-the-art performance for the semantic segmentation task.

show abstract

Section: B Rgb-d Semantic Segmentationmentioning

confidence: 99%

Pixel Difference Convolutional Network for RGB-D Semantic Segmentation

Yang¹,

Bai²,

Sun³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…CaiT [15] introduces a class-attention layer which can separate the contradictory objectives of guiding attention. In addition, there are many works [16,17,18,19,20] focusing on the application of vision transformers.…”

Section: A Vision Transformersmentioning

confidence: 99%

Quadcopter Drone for Vision-Based Autonomous Target Following

et al. 2023

View full text Add to dashboard Cite

Unmanned aerial vehicles (UAVs) are becoming popular in various applications. However, there are still challenging issues to be tackled, such as effective obstacle avoidance, target identification within a crowd, and specific target tracking. This paper focuses on dynamic target following and obstacle avoidance to realize a prototype of a quadcopter drone to serve as an autonomous object follower. An adaptive target identification system is proposed to recognize the specific target in the complicated background. For obstacle avoidance during flight, we introduce an idea of space detection and use it to develop a so-called contour and spiral convolution space detection (CASCSD) algorithm to evade obstacles. Thanks to the low architecture complexity, it is appropriate for implementation on onboard flight control systems. The target prediction is integrated with fuzzified flight control to fulfill an autonomous target tracker. When this series of technical research and development is completed, this system can be used for applications such as personal security guard and criminal detection systems.

show abstract

“…17 There are many scenarios for multitask learning applications, such as pixel prediction, sentiment analysis, hotspot detection, and audio pattern recognition. 18–22…”

Section: Introductionmentioning

confidence: 99%

“…17 There are many scenarios for multitask learning applications, such as pixel prediction, sentiment analysis, hotspot detection, and audio pattern recognition. [18][19][20][21][22] This study implements a multiforward reaction prediction transformer (MFRPT) and retro-forward reaction prediction transformer (RFRPT) to demonstrate the feasibility of multitasking in chemical reaction prediction. Several classical lowresource datasets involving Baeyer-Villiger, Heck, and Chan-Lam reactions are used.…”

Section: Introductionmentioning

confidence: 99%