Discrete Spherical Image Representation for CNN-Based Inclination Estimation

Abstract

How an image is represented as the input of a convolutional neural network (CNN) is important because this input directly influences the performance of the CNN. In this paper, we investigate the representation of spherical images by focusing on the inclination estimation of a spherical camera. Unlike other approaches to CNN-based inclination estimation, a spherical image is represented as a geodesicdivision-based discrete spherical image (DSI) that is obtained by sampling a sphere as uniformly as possible. The input of the CNN is a single image that consists of five parallelograms flattened from a regular icosahedron. To demonstrate the advantage of the proposed method, comparative experiments are conducted with two other spherical image representations, namely, equirectangular projection (ERP) and cubemap projection (CMP). The experimental results show that the proposed method using a geodesic-division-based discrete spherical image as the CNN input obtains the best performance-better than that of the cubemap and far superior to that of the equirectangular image. The effect of the image representations used becomes more significant as the relative inclination decreases. Moreover, comparative experiments are conducted using the state-of-the-art methods for spherical camera inclination compensation to further illustrate the superiority of the DSI representation. Consequently, the proposed method provides an important reference for the development of CNNs intended for spherical images.

Upright Adjustment With Graph Convolutional Networks

“…Jung et al [3] chose the equirectangular projection, which serves to be the most popular choice. Yu et al [5] investigated more accurate projection methods and proposed the discrete spherical image representation.…”

“…1 illustrates an example of the upright adjustment. Recently, few studies have been conducted on upright adjustment of 360 • images based on the deep learning algorithm [3,4,5] and have adopted the convolutional neural net- In this paper, we investigate a way to process the 360 • image in its own natural shape (sphere). For processing of spherical data, we adopt the graph convolutional networks (GCN).…”

Photometric Stereo with Twin-Fisheye Cameras

“…Robotics and consumer electronics have also recently benefited from the attractive features of twin-fisheye sensors, e.g. for image-based pose estimation [26,27].…”

SilhoNet-Fisheye: Adaptation of A ROI Based Object Pose Estimation Network to Monocular Fisheye Images

“…Closely related to fisheye image processing is the extensive body of work on omni-directional imaging, as both fisheye and omni-directional image distortions can be represented on a sphere. Beyond naively applying CNNs directly to a flattened equirectangular projection of an omni-directional image, which has been shown to suffer from the nonlinear distortions of the spherical mapping to the plane and attain sub-optimal performance [7], the methods of dealing with omni-directional distortions can be roughly categorized under three approaches: generating multiple perspective projections from the sphere, such as cube map, and processing each projection separately through the CNN [8]; adapting the kernel sampling locations based on a spherical distortion model or a learned mapping [9]- [12]; re-sampling the spherical image based on a uniform sampling geometry such as the icosahedron, and processing the spherical representation with specialized convolution operations [13]- [16]; or transforming the spherical feature signals and convolution operations into the spectral domain, typically by representation of the spherical image as a graph [17]- [19]. Methods that operate on multiple perspective projections suffer from discontinuities at the projection borders, due to variance in feature appearance on different tangent plane mappings.…”