Self-Supervised Pretraining for RGB-D Salient Object Detection

Wang

et al. 2023

Sensors

Detecting salient objects in complicated scenarios is a challenging problem. Except for semantic features from the RGB image, spatial information from the depth image also provides sufficient cues about the object. Therefore, it is crucial to rationally integrate RGB and depth features for the RGB-D salient object detection task. Most existing RGB-D saliency detectors modulate RGB semantic features with absolution depth values. However, they ignore the appearance contrast and structure knowledge indicated by relative depth values between pixels. In this work, we propose a depth-induced network (DIN) for RGB-D salient object detection, to take full advantage of both absolute and relative depth information, and further, enforce the in-depth fusion of the RGB-D cross-modalities. Specifically, an absolute depth-induced module (ADIM) is proposed, to hierarchically integrate absolute depth values and RGB features, to allow the interaction between the appearance and structural information in the encoding stage. A relative depth-induced module (RDIM) is designed, to capture detailed saliency cues, by exploring contrastive and structural information from relative depth values in the decoding stage. By combining the ADIM and RDIM, we can accurately locate salient objects with clear boundaries, even from complex scenes. The proposed DIN is a lightweight network, and the model size is much smaller than that of state-of-the-art algorithms. Extensive experiments on six challenging benchmarks, show that our method outperforms most existing RGB-D salient object detection models.

Section: Results and Analysismentioning

confidence: 99%

Section: Ablation Studiesmentioning

confidence: 99%

See 1 more Smart Citation

Absolute and Relative Depth-Induced Network for RGB-D Salient Object Detection

Wang

et al. 2023

Sensors

Third International Conference on Computer Vision and Data Mining (ICCVDM 2022)

“…The proposed method was compared with 15 significant object detection methods proposed in recent years: 3DCNN [15], HAIN [20], DSAM [19], ICNet [16], JLDCF [14], UCNet [7], SSF [31], SM2A [32], CMW [17], PGAR [33], SSP [34], ASSR [13], DPAG [9], D3Net [6], DMRA [10], and DLRF [8].The results of the method for the four test datasets are listed in Table 1. As F max , S m , and E m become higher, the result improves.…”

Section: Comparison With the State-of-the-artmentioning

confidence: 99%

RGB-D salient object detection based on multimodal feature information fusion

Meng

Yuan

Shi

et al. 2023

The key to RGB-D salient object detection is the effective fusion of the different modal features of RGB and depth maps. This study proposes an RGB-D salient object detection method based on multimodal feature information fusion. First, in the encoding stage, essential features from the depth map were extracted using the spatial and channel attention modules and then merged with RGB feature information to improve the expression ability of salient objects. Second, in the decoding stage, a multimodal and multilevel feature fusion module and a global context-feature guidance module were proposed to optimize the detection effect of the network on the salient objects of missing detection and error detection, which can more accurately decode the spatial structure information of multiple objects and small objects. Compared with 15 other deep learning detection methods, the experimental results on four datasets show that our method overcomes the comparison methods on multiple evaluation metrics.

“…In contrast with RGB images, without being limited by the similar indiscernible appearance, Depth maps have a foreground object that is distinct from the background, which theoretically leads to better segmentation performance, compared to RGB images only. In fact, Depth contours are cleaner than RGB contours and tend to better describe the edge information of objects for better separation of objects [62]. In contrast, RGB images are frequently utilized for providing global information [14].…”

Section: Introductionmentioning

confidence: 99%

Pixel Difference Convolutional Network for RGB-D Semantic Segmentation

Yang¹,

Bai²,

Sun³

et al. 2023

Preprint

RGB-D semantic segmentation can be advanced with convolutional neural networks due to the availability of Depth data. Although objects cannot be easily discriminated by just the 2D appearance, with the local pixel difference and geometric patterns in Depth, they can be well separated in some cases. Considering the fixed grid kernel structure, CNNs are limited to lack the ability to capture detailed, finegrained information and thus cannot achieve accurate pixellevel semantic segmentation. To solve this problem, we propose a Pixel Difference Convolutional Network (PDCNet) to capture detailed intrinsic patterns by aggregating both intensity and gradient information in the local range for Depth data and global range for RGB data, respectively. Precisely, PDCNet consists of a Depth branch and an RGB branch. For the Depth branch, we propose a Pixel Difference Convolution (PDC) to consider local and detailed geometric information in Depth data via aggregating both intensity and gradient information. For the RGB branch, we contribute a lightweight Cascade Large Kernel (CLK) to extend PDC, namely CPDC, to enjoy global contexts for RGB data and further boost performance. Consequently, both modal data's local and global pixel differences are seamlessly incorporated into PDCNet during the information propagation process. Experiments on two challenging benchmark datasets, i.e., NYUDv2 and SUN RGB-D reveal that our PDCNet achieves state-of-the-art performance for the semantic segmentation task.