360$$^{\circ }$$ Camera Alignment via Segmentation

Davidson, Benjamin; Alvi, Mohsan; Henriques, João F.

doi:10.1007/978-3-030-58604-1_35

Cited by 14 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Calibration methods for only extrinsic parameters have been proposed that are aimed at narrow view cameras [19,32,38,39,44,45] and panoramic 360 • images [10]. These methods cannot calibrate intrinsic parameters, that is, they cannot remove distortion.…”

Section: Related Workmentioning

confidence: 99%

Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion

Wakai¹,

Sato²,

Ishii³

et al. 2021

Preprint

View full text Add to dashboard Cite

Although recent learning-based calibration methods can predict extrinsic and intrinsic camera parameters from a single image, the accuracy of these methods is degraded in fisheye images. This degradation is caused by mismatching between the actual projection and expected projection. To address this problem, we propose a generic camera model that has the potential to address various types of distortion. Our generic camera model is utilized for learning-based methods through a closed-form numerical calculation of the camera projection. Simultaneously to recover rotation and fisheye distortion, we propose a learning-based calibration method that uses the camera model. Furthermore, we propose a loss function that alleviates the bias of the magnitude of errors for four extrinsic and intrinsic camera parameters. Extensive experiments demonstrated that our proposed method outperformed conventional methods on two largescale datasets and images captured by off-the-shelf fisheye cameras. Moreover, we are the first researchers to analyze the performance of learning-based methods using various types of projection for off-the-shelf cameras.

show abstract

Section: Related Workmentioning

confidence: 99%

Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion

Wakai¹,

Sato²,

Ishii³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A nonlevel 360 • image completely breaks the sense of realism for virtual reality users and leads to an unpleasant immersive experience and even severe user sickness [18]. In case the 360 • images are not upright, they can be leveled using [14], [15]. Another consequence of the rendering modes is that the images should be of a very high resolution (up to 10K) so that the HMD visualization remains of a sufficiently good quality.…”

Section: A Problem Settingsmentioning

confidence: 99%

“…This construction results from the exploitation of a sampling that is both uniform and oriented with respect to the north pole. It also takes advantage of the fact that omnidirectional images are usually level or registered 1 [14], [15]. Moreover, we propose an iterative construction of this convolution with a dedicated aggregation such that the kernel can be learned for every pixel neighborhood size.…”

Section: Introductionmentioning

confidence: 99%

OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression

Bidgoli¹,

Azevedo²,

Maugey³

et al. 2021

Preprint

View full text Add to dashboard Cite

State-of-the-art 2D image compression schemes rely on the power of convolutional neural networks (CNNs). Although CNNs offer promising perspectives for 2D image compression, extending such models to omnidirectional images is not straightforward. First, omnidirectional images have specific spatial and statistical properties that can not be fully captured by current CNN models. Second, basic mathematical operations composing a CNN architecture, e.g., translation and sampling, are not welldefined on the sphere. In this paper, we study the learning of representation models for omnidirectional images and propose to use the properties of HEALPix uniform sampling of the sphere to redefine the mathematical tools used in deep learning models for omnidirectional images. In particular, we: i) propose the definition of a new convolution operation on the sphere that keeps the high expressiveness and the low complexity of a classical 2D convolution; ii) adapt standard CNN techniques such as stride, iterative aggregation, and pixel shuffling to the spherical domain; and then iii) apply our new framework to the task of omnidirectional image compression. Our experiments show that our proposed on-the-sphere solution leads to a better compression gain that can save 13.7% of the bit rate compared to similar learned models applied to equirectangular images. Also, compared to learning models based on graph convolutional networks, our solution supports more expressive filters that can preserve high frequencies and provide a better perceptual quality of the compressed images. Such results demonstrate the efficiency of the proposed framework, which opens new research venues for other omnidirectional vision tasks to be effectively implemented on the sphere manifold.

show abstract

“…We expect, however, the images to be approximately gravity-aligned, as in all common datasets available [23][24][25][26][27][28]. This condition is a defacto standard for practically all indoor static and mobile acquisition setups, as they are equipped with automatic georeferencing and alignment systems [7,8,[29][30][31]. It is worth noting that we can accommodate for large tolerances in gravity alignment.…”

Section: Introductionmentioning

confidence: 99%

Deep panoramic depth prediction and completion for indoor scenes

Pintore,

Almansa,

Sanchez

et al. 2024

Comp. Visual Media

View full text Add to dashboard Cite

We introduce a novel end-to-end deep-learning solution for rapidly estimating a dense spherical depth map of an indoor environment. Our input is a single equirectangular image registered with a sparse depth map, as provided by a variety of common capture setups. Depth is inferred by an efficient and lightweight single-branch network, which employs a dynamic gating system to process together dense visual data and sparse geometric data. We exploit the characteristics of typical man-made environments to efficiently compress multi-resolution features and find short- and long-range relations among scene parts. Furthermore, we introduce a new augmentation strategy to make the model robust to different types of sparsity, including those generated by various structured light sensors and LiDAR setups. The experimental results demonstrate that our method provides interactive performance and outperforms state-of-the-art solutions in computational efficiency, adaptivity to variable depth sparsity patterns, and prediction accuracy for challenging indoor data, even when trained solely on synthetic data without any fine tuning.

show abstract

360$$^{\circ }$$ Camera Alignment via Segmentation

Cited by 14 publications

References 30 publications

Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion

Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion

OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression

Deep panoramic depth prediction and completion for indoor scenes

Contact Info

Product

Resources

About