UniFuse: Unidirectional Fusion for 360° Panorama Depth Estimation

Jiang, Hualie; Sheng, Zhe; Zhu, Siyu; Dong, Zilong; Huang, Rui

doi:10.1109/lra.2021.3058957

Cited by 93 publications

(83 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The reason is that our dataset is actually captured by real panoramic cameras, so were our training dataset. While the data in Matterport3D we got in the same way as Unifuse [54] told in their paper, was more or less rendered images and lost a lot of distant information in the RGB image. The case 3 was a totally failure case.…”

Section: )Umde Depth Estimation Results and Analyzationsmentioning

confidence: 99%

3D Reconstruction with Spherical Cameras

et al. 2024

View full text Add to dashboard Cite

The goal of image-based 3D reconstruction is to establish a high-quality 3D expression from images. In order to achieve a high-resolution and real-time 3D model, inspired by the open source COLMAP, we propose a novel framework (3DMAP) to reconstruct 3D scenes based on spherical cameras. Unlike traditional methods which focus on building a 3D plane by Poisson distribution function, our method illustrates the key processes of 3D reconstruction: locating the camera based on global feature, estimating the scene's relative depth from monocular panoramic images, and obtaining a high-quality 3D surface reconstruction. In the camera locating part, we use a global descriptor augmentation model to build a labeled panorama dataset GDAP, in which the images are captured by our designed spherical cameras; In the depth estimation part, we propose a new network UMDE that can estimate the depth of both indoor and outdoor scenes; Finally, in the 3D surface reconstruction section, we turn the reconstruction problem to a graph optimization problem, called GraphFit, in which, we optimize the point clouds with s-t graph and smoothing method successively. We conduct experiments on our own dataset to demonstrate the effectiveness of our proposed 3DMAP framework. Experimental results show that our 3DMAP has achieved good evaluation scores and visual effects.

show abstract

Section: )Umde Depth Estimation Results and Analyzationsmentioning

confidence: 99%

3D Reconstruction with Spherical Cameras

et al. 2024

View full text Add to dashboard Cite

show abstract

“…Wang et al [35] propose a trainable deep network, 360SD-Net, designed for 360 • stereo images. The approaches of Jiang et al [36] and Wang et al [37] fuse information extracted by both the equirectangular and the cube map projections of a single omnidirectional image within a fully convolutional network framework. Jin et al [38] propose a learning-based depth estimation framework based on the geometric structure of the scene.…”

Section: Depth Estimationmentioning

confidence: 99%

A Dataset of Annotated Omnidirectional Videos for Distancing Applications

et al. 2021

View full text Add to dashboard Cite

Omnidirectional (or 360°) cameras are acquisition devices that, in the next few years, could have a big impact on video surveillance applications, research, and industry, as they can record a spherical view of a whole environment from every perspective. This paper presents two new contributions to the research community: the CVIP360 dataset, an annotated dataset of 360° videos for distancing applications, and a new method to estimate the distances of objects in a scene from a single 360° image. The CVIP360 dataset includes 16 videos acquired outdoors and indoors, annotated by adding information about the pedestrians in the scene (bounding boxes) and the distances to the camera of some points in the 3D world by using markers at fixed and known intervals. The proposed distance estimation algorithm is based on geometry facts regarding the acquisition process of the omnidirectional device, and is uncalibrated in practice: the only required parameter is the camera height. The proposed algorithm was tested on the CVIP360 dataset, and empirical results demonstrate that the estimation error is negligible for distancing applications.

show abstract

“…Panoramic Image Processing. Panoramic (aka 360°or spherical) images have been used in a wide variety of application areas, including depth estimation [19,20,24,34,41,42,44,50,51], room layout estimation [12,14,21,34,40,43,47], semantic segmentation [25,34,46,48], novel-view synthesis [5,16,28,45] and so on. The key problem when working with spherical images is to project them onto a regular 2D pixel grid for easy processing, as any projection introduces some kind of distortion -similar to maps of the world.…”

Section: Related Workmentioning

confidence: 99%

360° Optical Flow using Tangent Images

Yuan¹,

Richardt²

2021

Preprint

View full text Add to dashboard Cite

Omnidirectional 360°images have found many promising and exciting applications in computer vision, robotics and other fields, thanks to their increasing affordability, portability and their 360°field of view. The most common format for storing, processing and visualising 360°images is equirectangular projection (ERP). However, the distortion introduced by the nonlinear mapping from 360°images to ERP images is still a barrier that holds back ERP images from being used as easily as conventional perspective images. This is especially relevant when estimating 360°optical flow, as the distortions need to be mitigated appropriately. In this paper, we propose a 360°optical flow method based on tangent images. Our method leverages gnomonic projection to locally convert ERP images to perspective images, and uniformly samples the ERP image by projection to a cubemap and regular icosahedron faces, to incrementally refine the estimated 360°flow fields even in the presence of large rotations. Our experiments demonstrate the benefits of our proposed method both quantitatively and qualitatively.

show abstract

UniFuse: Unidirectional Fusion for 360° Panorama Depth Estimation

Cited by 93 publications

References 37 publications

3D Reconstruction with Spherical Cameras

3D Reconstruction with Spherical Cameras

A Dataset of Annotated Omnidirectional Videos for Distancing Applications

360° Optical Flow using Tangent Images

Contact Info

Product

Resources

About