Toward Real-world Panoramic Image Enhancement

Zhang, Yupeng; Zhang, Hengzhi; Li, Daojing; Liu, Liyan; Hong, Yong-Geun; Wang, Wei; Suitoh, Hiroshi; Odamaki, Makoto

doi:10.1109/cvprw50498.2020.00322

Cited by 8 publications

(2 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is stated that users have almost the same behavior and navigation when wearing an HMD as when they watch 360-degree videos. Zhang et al [24] studied a quality enhancement of 360-degree images using deep neural networks based on GANs without changing the image resolution. They designed a compact network employing a multi-frequency structure with compressed Residual-in-Residual Dense Blocks and convolution layers from each dense block.…”

Section: Literature Reviewmentioning

confidence: 99%

Generation of a Panorama Compatible with the JPEG 360 International Standard Using a Single PTZ Camera

2021

View full text Add to dashboard Cite

Recently, the JPEG working group (ISO/IEC JTC1 SC29 WG1) developed an international standard, JPEG 360, that specifies the metadata and functionalities for saving and sharing 360-degree images efficiently to create a more realistic environment in various virtual reality services. We surveyed the metadata formats of existing 360-degree images and compared them to the JPEG 360 metadata format. We found that existing omnidirectional cameras and stitching software packages use formats that are incompatible with the JPEG 360 standard to embed metadata in JPEG image files. This paper proposes an easy-to-use tool for embedding JPEG 360 standard metadata for 360-degree images in JPEG image files using a JPEG-defined box format: the JPEG universal metadata box format. The proposed implementation will help 360-degree cameras and software vendors provide immersive services to users in a standardized manner for various markets, such as entertainment, education, professional training, navigation, and virtual and augmented reality applications. We also propose and develop an economical JPEG 360 standard compatible panoramic image acquisition system from a single PTZ camera with a special-use case of a wide field of view image of a conference or meeting. A remote attendee of the conference/meeting can see the realistic and immersive environment through our PTZ panorama in virtual reality.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Generation of a Panorama Compatible with the JPEG 360 International Standard Using a Single PTZ Camera

2021

View full text Add to dashboard Cite

show abstract

“…[29] leveraged GAN to super-resolve an ODI by applying WS-SSIM [47]. [46] also proposed the GAN-based framework employing multi-frequency structures to enhance the panoramic image quality up to the high-end camera quality. [23] focused on the 360 • image SR utilizing single-frame and multi-frame joint learning and a loss function weighted differently along the latitude.…”

Section: Related Workmentioning

confidence: 99%

SphereSR: 360° Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation

Yoon¹,

Chung²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

The 360 • imaging has recently gained great attention; however, its angular resolution is relatively lower than that of a narrow field-of-view (FOV) perspective image as it is captured by using fisheye lenses with the same sensor size. Therefore, it is beneficial to super-resolve a 360 • image. Some attempts have been made but mostly considered the equirectangular projection (ERP) as one of the way for 360 • image representation despite of latitude-dependent distortions. In that case, as the output high-resolution (HR) image is always in the same ERP format as the lowresolution (LR) input, another information loss may occur when transforming the HR image to other projection types. In this paper, we propose SphereSR, a novel framework to generate a continuous spherical image representation from an LR 360 • image, aiming at predicting the RGB values at given spherical coordinates for super-resolution with an arbitrary 360 • image projection. Specifically, we first propose a feature extraction module that represents the spherical data based on icosahedron and efficiently extracts features on the spherical surface. We then propose a spherical local implicit image function (SLIIF) to predict RGB values at the spherical coordinates. As such, SphereSR flexibly reconstructs an HR image under an arbitrary projection type. Experiments on various benchmark datasets show that our method significantly surpasses existing methods.

show abstract