Neural Window Fully-connected CRFs for Monocular Depth Estimation

Yuan, Weihao; Gu, Xiaodong; Dai, Zuozhuo; Zhu, Siyu; Tan, Ping

doi:10.1109/cvpr52688.2022.00389

Cited by 196 publications

(82 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…NeWCRFs [34] Ours GT Figure 1: We observe that depth boundaries in state-of-theart [34] align well with object boundaries, but the depth label is often incorrect. Note the confusion for the middle pillow in the first row and the bed in the second row.…”

Section: Inputmentioning

confidence: 91%

“…The feature pyramidal-based decoder mitigates the issue by fusing low-resolution, semantically rich decoder features with the higher resolution but semantically weaker encoder features via a top-down path-way and lateral connections called skip connections [18]. Inline with the recent success of transformers, many latest works have used a self-attention based architectures for MDE [1,32,34]. Self-attention increases the receptive field and allows to capture long-range dependencies in feature maps.…”

Section: Inputmentioning

confidence: 99%

“…Practically, it is challenging to use self-attention for high-resolution feature maps due to memory and computational constraints. Hence, the current SOTA [34] uses window based attention using Swin transformer-based encoder backbone [19] to improve efficiency. We observe that SOTA [1,34] techniques are highly accurate in aligning depth edges with the object boundaries.…”

Section: Inputmentioning

confidence: 99%

“…Hence, the current SOTA [34] uses window based attention using Swin transformer-based encoder backbone [19] to improve efficiency. We observe that SOTA [1,34] techniques are highly accurate in aligning depth edges with the object boundaries. However, there exists a confusion in giving the depth label to a pixel (c.f.…”

Section: Inputmentioning

confidence: 99%

See 3 more Smart Citations

Attention Attention Everywhere: Monocular Depth Prediction with Skip Attention

Agarwal¹,

Arora²

2022

Preprint

View full text Add to dashboard Cite

Monocular Depth Estimation (MDE) aims to predict pixel-wise depth given a single RGB image. For both, the convolutional as well as the recent attention-based models, encoder-decoder-based architectures have been found to be useful due to the simultaneous requirement of global context and pixel-level resolution. Typically, a skip connection module is used to fuse the encoder and decoder features, which comprises of feature map concatenation followed by a convolution operation. Inspired by the demonstrated benefits of attention in a multitude of computer vision problems, we propose an attention-based fusion of encoder and decoder features. We pose MDE as a pixel query refinement problem, where coarsest-level encoder features are used to initialize pixel-level queries, which are then refined to higher resolutions by the proposed Skip Attention Module (SAM). We formulate the prediction problem as ordinal regression over the bin centers that discretize the continuous depth range and introduce a Bin Center Predictor (BCP) module that predicts bins at the coarsest level using pixel queries. Apart from the benefit of image adaptive depth binning, the proposed design helps learn improved depth embedding in initial pixel queries via direct supervision from the ground truth. Extensive experiments on the two canonical datasets, NYUV2 and KITTI, show that our architecture outperforms the state-of-the-art by 5.3% and 3.9%, respectively, along with an improved generalization performance by 9.4% on the SUNRGBD dataset. Code is available at https://github.com/ ashutosh1807/PixelFormer.git.

show abstract

Section: Inputmentioning

confidence: 91%

Section: Inputmentioning

confidence: 99%

Section: Inputmentioning

confidence: 99%

Section: Inputmentioning

confidence: 99%

See 2 more Smart Citations

Attention Attention Everywhere: Monocular Depth Prediction with Skip Attention

Agarwal¹,

Arora²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, models using a classification model to boost the performance of depth estimation, such as binsformer [25] or adpative bins [26], perform well with the monocular depth estimation. However, Neural window FC-CRFs (NeWCRF) reaches the same performance by applying Conditional Random Field (CRF) on the decoder part to regress the depth map by utilizing fully-connected CRFs on each split image part (window) [27]. Therefore, we chose the Swin Transformer with NeWCRF decoder among the state of the art models for monocular depth estimation.…”

Section: A Algorithms For Shape Reconstructionmentioning

confidence: 99%

DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction

Do¹,

Kennedy²

2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Collaborative robots stand to have an immense impact both on human welfare in domestic service applications and industrial superiority in advanced manufacturing requires dexterous assembly. The outstanding challenge is providing robotic fingertips with a physical design that makes them adept at performing dexterous tasks that require high-resolution, calibrated shape reconstruction and force sensing. In this work, we present DenseTact 2.0, an optical-tactile sensor capable of visualizing the deformed surface of a soft fingertip and using that image in a neural network to perform both calibrated shape reconstruction and 6-axis wrench estimation. We demonstrate the sensor accuracy of 0.3633mm per pixel for shape reconstruction, 0.410N for forces, 0.387mmN •m for torques, and the ability to calibrate new fingers through transfer learning, achieving comparable performance that trained more than four times faster with only 12% of the dataset size.

show abstract

LocalBins: Improving Depth Estimation by Learning Local Distributions

Bhat

Alhashim

Wonka

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We propose a novel architecture for depth estimation from a single image. The architecture itself is based on the popular encoderdecoder architecture that is frequently used as a starting point for all dense regression tasks. We build on AdaBins which estimates a global distribution of depth values for the input image and evolve the architecture in two ways. First, instead of predicting global depth distributions, we predict depth distributions of local neighborhoods at every pixel. Second, instead of predicting depth distributions only towards the end of the decoder, we involve all layers of the decoder. We call this new architecture LocalBins. Our results demonstrate a clear improvement over the state-of-the-art in all metrics on the NYU-Depth V2 dataset. Code and pretrained models will be made publicly available. 3

show abstract

Neural Window Fully-connected CRFs for Monocular Depth Estimation

Cited by 196 publications

References 25 publications

Attention Attention Everywhere: Monocular Depth Prediction with Skip Attention

Attention Attention Everywhere: Monocular Depth Prediction with Skip Attention

DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction

LocalBins: Improving Depth Estimation by Learning Local Distributions

Contact Info

Product

Resources

About