Hierarchical Object Relationship Constrained Monocular Depth Estimation.

Li, Shuai; Shi, Jiaying; Song, Wenfeng; Hao, Aimin; Qin, Hong

doi:10.1016/j.patcog.2021.108116

Cited by 12 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite current limitations in depth estimation and inpainting, the extraordinarily rapid pace of recent progress [RLH ∗ 20; SSKH20; MDM ∗ 21; WLS ∗ 21; JPY21; LSS ∗ 21; JCS ∗ 21; SPR21; LWH ∗ 21; ZBSA21; ZLLP21; MZH ∗ 21] suggest these areas will improve in the coming years. Our goal is to provide new, useful ways to edit photographs given these techniques.…”

Section: Discussionmentioning

confidence: 99%

ZoomShop: Depth‐Aware Editing of Photographic Composition

Liu

Agrawala

DiVerdi

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

Figure 1: Using ZoomShop to edit a photograph [Tha16]. (a) An image of mountains (background) framed by trees (foreground). The user's goal is to make the boat bigger while keeping the framing of foreground trees. (b) Zooming in and cropping scales up the boat, but cuts out most of the trees and shore. (c) Left: With ZoomShop users can select depth ranges (yellow and green) and independently adjust each region while maintaining scene structure. Disoccluded and stretched pixels are shown in cyan. Right: Cyan pixels from image are manually inpainted using Photoshop's Content-Aware Fill. We show the top-down view volume and boundary curve of each camera above the corresponding images in a, b, and c.

show abstract

Section: Discussionmentioning

confidence: 99%

ZoomShop: Depth‐Aware Editing of Photographic Composition

Liu

Agrawala

DiVerdi

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…To train the network, they devised a function to measure projected consistency loss based on the transformation relationship between target size and depth. Li et al [ 21 ] utilized the relationships between neighboring objects and the relationships among all objects in a scene for MDE. The method can produce accurate depth maps for indoor scenes.…”

Section: Related Workmentioning

confidence: 99%

Deep Learning-Based Monocular 3D Object Detection with Refinement of Depth Information

Zhu

Li³

et al. 2022

Sensors

View full text Add to dashboard Cite

Recently, the research on monocular 3D target detection based on pseudo-LiDAR data has made some progress. In contrast to LiDAR-based algorithms, the robustness of pseudo-LiDAR methods is still inferior. After conducting in-depth experiments, we realized that the main limitations are due to the inaccuracy of the target position and the uncertainty in the depth distribution of the foreground target. These two problems arise from the inaccurate depth estimation. To deal with the aforementioned problems, we propose two innovative solutions. The first is a novel method based on joint image segmentation and geometric constraints, used to predict the target depth and provide the depth prediction confidence measure. The predicted target depth is fused with the overall depth of the scene and results in the optimal target position. For the second, we utilize the target scale, normalized with the Gaussian function, as a priori information. The uncertainty of depth distribution, which can be visualized as long-tail noise, is reduced. With the refined depth information, we convert the optimized depth map into the point cloud representation, called a pseudo-LiDAR point cloud. Finally, we input the pseudo-LiDAR point cloud to the LiDAR-based algorithm to detect the 3D target. We conducted extensive experiments on the challenging KITTI dataset. The results demonstrate that our proposed framework outperforms various state-of-the-art methods by more than 12.37% and 5.34% on the easy and hard settings of the KITTI validation subset, respectively. On the KITTI test set, our framework also outperformed state-of-the-art methods by 5.1% and 1.76% on the easy and hard settings, respectively.

show abstract

“…These works generally refer to MDE denoising [55,56] and MDE refinement [57,58]. More recently, new depth completion methods, such as depth superresolution [59][60][61][62] using subpixel convolutional layers, and multi-resolution fusion [63,64] using content-adaptive depth merging networks, have also shown fairly good impacts on MDE, producing high-quality depth maps with effective spatial resolutions that are very close to those of the input images.…”

Section: Previous Workmentioning

confidence: 99%

MADNet 2.0: Pixel-Scale Topography Retrieval from Single-View Orbital Imagery of Mars Using Deep Learning

et al. 2021

View full text Add to dashboard Cite

The High-Resolution Imaging Science Experiment (HiRISE) onboard the Mars Reconnaissance Orbiter provides remotely sensed imagery at the highest spatial resolution at 25–50 cm/pixel of the surface of Mars. However, due to the spatial resolution being so high, the total area covered by HiRISE targeted stereo acquisitions is very limited. This results in a lack of the availability of high-resolution digital terrain models (DTMs) which are better than 1 m/pixel. Such high-resolution DTMs have always been considered desirable for the international community of planetary scientists to carry out fine-scale geological analysis of the Martian surface. Recently, new deep learning-based techniques that are able to retrieve DTMs from single optical orbital imagery have been developed and applied to single HiRISE observational data. In this paper, we improve upon a previously developed single-image DTM estimation system called MADNet (1.0). We propose optimisations which we collectively call MADNet 2.0, which is based on a supervised image-to-height estimation network, multi-scale DTM reconstruction, and 3D co-alignment processes. In particular, we employ optimised single-scale inference and multi-scale reconstruction (in MADNet 2.0), instead of multi-scale inference and single-scale reconstruction (in MADNet 1.0), to produce more accurate large-scale topographic retrieval with boosted fine-scale resolution. We demonstrate the improvements of the MADNet 2.0 DTMs produced using HiRISE images, in comparison to the MADNet 1.0 DTMs and the published Planetary Data System (PDS) DTMs over the ExoMars Rosalind Franklin rover’s landing site at Oxia Planum. Qualitative and quantitative assessments suggest the proposed MADNet 2.0 system is capable of producing pixel-scale DTM retrieval at the same spatial resolution (25 cm/pixel) of the input HiRISE images.

show abstract

Hierarchical Object Relationship Constrained Monocular Depth Estimation.

Cited by 12 publications

References 17 publications

ZoomShop: Depth‐Aware Editing of Photographic Composition

ZoomShop: Depth‐Aware Editing of Photographic Composition

Deep Learning-Based Monocular 3D Object Detection with Refinement of Depth Information

MADNet 2.0: Pixel-Scale Topography Retrieval from Single-View Orbital Imagery of Mars Using Deep Learning

Contact Info

Product

Resources

About