Distortion-Tolerant Monocular Depth Estimation on Omnidirectional Images Using Dual-Cubemap

Shen, Zhijie; Lin, Chunyu; Nie, Lang; Liao, Kang; Zhao, Yao

doi:10.1109/icme51207.2021.9428385

Cited by 14 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated by previous works [7,16], we first polymerize the most relevant features via a virtue tangent plane on the panoramic sphere projection. Based on the projection formula [16,17,4] (we discuss it in detail in the supplementary material), we can get the distortion-free sampling coordinates p ∈ R H×W ×9×2 (height, width, number of points, number of coordinates) in the 2D feature maps. The learnable offsets ∆p are employed to adjust the sampling locations adaptively.…”

Section: Cross-scale Distortion Awarenessmentioning

confidence: 99%

Disentangling Orthogonal Planes for Indoor Panoramic Room Layout Estimation with Cross-Scale Distortion Awareness

Shen¹,

Zheng²,

Lin³

et al. 2023

Preprint

View full text Add to dashboard Cite

Based on the Manhattan World assumption, most existing indoor layout estimation schemes focus on recovering layouts from vertically compressed 1D sequences. However, the compression procedure confuses the semantics of different planes, yielding inferior performance with ambiguous interpretability.To address this issue, we propose to disentangle this 1D representation by pre-segmenting orthogonal (vertical and horizontal) planes from a complex scene, explicitly capturing the geometric cues for indoor layout estimation. Considering the symmetry between the floor boundary and ceiling boundary, we also design a soft-flipping fusion strategy to assist the pre-segmentation. Besides, we present a feature assembling mechanism to effectively integrate shallow and deep features with distortion distribution awareness. To compensate for the potential errors in pre-segmentation, we further leverage triple attention to reconstruct the disentangled sequences for better performance. Experiments on four popular benchmarks demonstrate our superiority over existing SoTA solutions, especially on the 3DIoU metric. The code is available at https://github.com/ zhijieshen-bjtu/DOPNet.

show abstract

Section: Cross-scale Distortion Awarenessmentioning

confidence: 99%

Disentangling Orthogonal Planes for Indoor Panoramic Room Layout Estimation with Cross-Scale Distortion Awareness

Shen¹,

Zheng²,

Lin³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…From a lightweight perspective, Jiang et al [17] improve Bifuse by designing an efficient fusion block and propose Unifuse. Besides, Shen et al [7] utilize a dual-cube fusion strategy to get rid of distortion. Compared with cubemap, there are less distortion and pixel loss in the tangent images, and Eder et al [18] propose leveraging tangent images to mitigate the spherical distortion.…”

Section: A Monocular 360°depth Estimationmentioning

confidence: 99%

“…For better supervision, we combine reverse Huber (or Berhu) [8], [16], [17], [39] loss and gradient loss [7] to keep the maximum value in the window, and set others to 0 4: end for 5: Set a non-overlapping sliding window with size of 1×7, 6: for each position in HF s f ct do 7:…”

Section: Objective Functionmentioning

confidence: 99%

“…end if 16: end for 17: return M s design our objective function as commonly used in previous works [7], [8]. The standard inverse Huber loss is:…”

Section: Objective Functionmentioning

confidence: 99%

“…Specifically, Chen et al [6] gain better performance by learning an effective structure representation. Shen et al [7], [8] improve the accuracy discernibly by constraining the depth edges. Apparently, the structure cue is reasonable.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Neural Contourlet Network for Monocular 360° Depth Estimation

Shen

Lin

Nie

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

For a monocular 360°image, depth estimation is a challenging because the distortion increases along the latitude. To perceive the distortion, existing methods devote to designing a deep and complex network architecture. In this paper, we provide a new perspective that constructs an interpretable and sparse representation for a 360°image. Considering the importance of the geometric structure in depth estimation, we utilize the contourlet transform to capture an explicit geometric cue in the spectral domain and integrate it with an implicit cue in the spatial domain. Specifically, we propose a neural contourlet network consisting of a convolutional neural network and a contourlet transform branch. In the encoder stage, we design a spatial-spectral fusion module to effectively fuse two types of cues. Contrary to the encoder, we employ the inverse contourlet transform with learned low-pass subbands and band-pass directional subbands to compose the depth in the decoder. Experiments on the three popular 360°panoramic image datasets demonstrate that the proposed approach outperforms the state-of-the-art schemes with faster convergence. Code is available at https://github.com/ zhijieshen-bjtu/Neural-Contourlet-Network-for-MODE.

show abstract

PanoFormer: Panorama Transformer for Indoor 360$$^{\circ }$$ Depth Estimation

Shen

Lin

Liao

et al. 2022

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Existing panoramic layout estimation solutions tend to recover room boundaries from a vertically compressed sequence, yielding imprecise results as the compression process often muddles the semantics between various planes. Besides, these datadriven approaches impose an urgent demand for massive data annotations, which are laborious and time-consuming. For the first problem, we propose an orthogonal plane disentanglement network (termed DOPNet) to distinguish ambiguous semantics. DOPNet consists of three modules that are integrated to deliver distortion-free, semantics-clean, and detail-sharp disentangled representations, which benefit the subsequent layout recovery. For the second problem, we present an unsupervised adaptation technique tailored for horizon-depth and ratio representations. Concretely, we introduce an optimization strategy for decision-level layout analysis and a 1D cost volume construction method for feature-level multi-view aggregation, both of which are designed to fully exploit the geometric consistency across multiple perspectives. The optimizer provides a reliable set of pseudo-labels for network training, while the 1D cost volume enriches each view with comprehensive scene information derived from other perspectives. Extensive experiments demonstrate that our solution outperforms other SoTA models on both monocular layout estimation and multi-view layout estimation tasks.

show abstract

Distortion-Tolerant Monocular Depth Estimation on Omnidirectional Images Using Dual-Cubemap

Cited by 14 publications

References 15 publications

Disentangling Orthogonal Planes for Indoor Panoramic Room Layout Estimation with Cross-Scale Distortion Awareness

Disentangling Orthogonal Planes for Indoor Panoramic Room Layout Estimation with Cross-Scale Distortion Awareness

Neural Contourlet Network for Monocular 360° Depth Estimation

PanoFormer: Panorama Transformer for Indoor 360$$^{\circ }$$ Depth Estimation

Contact Info

Product

Resources

About