TransDSSL: Transformer Based Depth Estimation via Self-Supervised Learning

Han, Daechan; Shin, Jeongmin; Kim, Namil; Hwang, Soonmin; Choi, Yukyung

doi:10.1109/lra.2022.3196781

Cited by 20 publications

(8 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chang et al [18] proposed an attentionbased up-sample block to compensate for the texture features and an attention supervision mechanism to provide further guidance for transformer layers. Han et al [19] proposed an attention-based decoder module to enhance fine details while keeping global context from the vision transformer.…”

Section: Vision Transformer For Depth Estimationmentioning

confidence: 99%

RCDPT: Radar-Camera Fusion Dense Prediction Transformer

Vandewalle

2023

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Recently, transformer networks have outperformed traditional deep neural networks in natural language processing and show a large potential in many computer vision tasks compared to convolutional backbones. In the original transformer, readout tokens are used as designated vectors for aggregating information from other tokens. However, the performance of using readout tokens in a vision transformer is limited. Therefore, we propose a novel fusion strategy to integrate radar data into a dense prediction transformer network by reassembling camera representations with radar representations. Instead of using readout tokens, radar representations contribute additional depth information to a monocular depth estimation model and improve performance. We further investigate different fusion approaches that are commonly used for integrating additional modality in a dense prediction transformer network. The experiments are conducted on the nuScenes dataset, which includes camera images, lidar, and radar data. The results show that our proposed method yields better performance than the commonly used fusion strategies and outperforms existing convolutional depth estimation models that fuse camera images and radar.

show abstract

Section: Vision Transformer For Depth Estimationmentioning

confidence: 99%

RCDPT: Radar-Camera Fusion Dense Prediction Transformer

Vandewalle

2023

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…2 showcases the technical pipeline of our Diffusion-Augmented Depth Prediction (DADP) framework. On autonomous driving scenarios only with sparse annotations [6,16], the key problem is to enforce regional and holistic spatial structures without the unreliable pose estimation in prior arts [14,19,20,51,55]. DADP contains two main components: a depth predictor and a noise predictor.…”

Section: Proposed Methods 31 Overviewmentioning

confidence: 99%

“…Self-supervised Depth Estimation. To enhance depth structures on driving scenes [6,16], prior arts [17,19,20,51,55] seek for the self-supervised paradigm. They jointly optimize a pose module and a depth module by photometric loss [12].…”

Section: Related Workmentioning

confidence: 99%

“…However, self-supervised methods [17,19,20,51,55] rely on pose estimation, which is unreliable and significantly limits their robustness. In contrast, our DADP achieves better robustness without pose estimation and multiple cameras.…”

Section: Related Workmentioning

confidence: 99%

“…datasets are in the self-supervised paradigm with pose estimation. We compare our DADP with previous self-supervised frameworks [14,17,19,20,51,55] in Table 2 and Fig. 6.…”

Section: Rgb Monodepth2 Ours(dpt) Surrounddepthmentioning

confidence: 99%

See 2 more Smart Citations

Celebrating the 70th Anniversary of School of Mechanical Science and Engineering of Huazhong University of Science & Technology

Wen

2022

IET Collab Intel Manufact

View full text Add to dashboard Cite

Depth estimation aims to predict dense depth maps. In autonomous driving scenes, sparsity of annotations makes the task challenging. Supervised models produce concave objects due to insufficient structural information. They overfit to valid pixels and fail to restore spatial structures. Self-supervised methods are proposed for the problem. Their robustness is limited by pose estimation, leading to erroneous results in natural scenes. In this paper, we propose a supervised framework termed Diffusion-Augmented Depth Prediction (DADP). We leverage the structural characteristics of diffusion model to enforce depth structures of depth models in a plug-and-play manner. An object-guided integrality loss is also proposed to further enhance regional structure integrality by fetching objective information. We evaluate DADP on three driving benchmarks and achieve significant improvements in depth structures and robustness. Our work provides a new perspective on depth estimation with sparse annotations in autonomous driving scenes. CCS CONCEPTS• Computing methodologies → Scene understanding.

show abstract