SphereSR: $360^{\circ}$ Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation

Yoon, Young-Ho; Chung, Inchul; Wang, Lin; Yoon, Kuk-Jin

doi:10.1109/cvpr52688.2022.00559

Cited by 32 publications

(18 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second category of methods to address distortion in 360°images is designing new projection methods. There are mainly four different types of projection: equirectangular projection, cube projection [3], icosahedron projection [15,38,40], and tangent projection [18]. Currently, equirectangular projection is still the most commonly used approach.…”

Section: Methods On 360°imagementioning

confidence: 99%

Materials Science at Beihang: A Special Issue of Materials Lab Dedicated to the 70th Anniversary of Beihang University

Zhao¹,

Guo²,

Jiang³

2022

MatLab

View full text Add to dashboard Cite

Beihang is the first university in China focusing on aeronautical and astronautical education and research, also covering diverse fields in the natural sciences, high technology, economics, management, the liberal arts, law, philosophy, foreign languages and education. School of Materials Science and Engineering at Beijing University was founded in 1954 as the Department of Aeronautical Metallurgy. In 2001, it was named School of Materials Science and Engineering, consisting

show abstract

Section: Methods On 360°imagementioning

confidence: 99%

Materials Science at Beihang: A Special Issue of Materials Lab Dedicated to the 70th Anniversary of Beihang University

Zhao¹,

Guo²,

Jiang³

2022

MatLab

View full text Add to dashboard Cite

show abstract

“…The first one utilizes priors of sphere-to-plane mapping, such as the LAU-Net (Deng et al 2021), which not only mitigates the issues of ERP but also is modelagnostic to most existing methods. However, the ability of this method is limited by the intrinsic characteristics of ERP (Yoon et al 2022). Additionally, it uses loss functions designed for 2D planar image SR, significantly impacting its performance for omnidirectional image SR by not considering the sampling issues of ERP.…”

Section: Densementioning

confidence: 99%

“…Additionally, it uses loss functions designed for 2D planar image SR, significantly impacting its performance for omnidirectional image SR by not considering the sampling issues of ERP. The other method designs spherical convolution for omnidirectional images, as see in SphereSR (Yoon et al 2022), where a new kernel weight is proposed to adapt to the inhomogeneous distributed sampling density and distortion across latitude in ERP. While achieving impressive performance in omnidirectional image super-resolution, this method suffers from high computational costs due to the repeated switching between spherical and 2D planar coordinates.…”

Section: Densementioning

confidence: 99%

Spherical Pseudo-Cylindrical Representation for Omnidirectional Image Super-resolution

Cai,

Li,

Ren

et al. 2024

AAAI

View full text Add to dashboard Cite

Omnidirectional images have attracted significant attention in recent years due to the rapid development of virtual reality technologies. Equirectangular projection (ERP), a naive form to store and transfer omnidirectional images, however, is challenging for existing two-dimensional (2D) image super-resolution (SR) methods due to its inhomogeneous distributed sampling density and distortion across latitude. In this paper, we make one of the first attempts to design a spherical pseudo-cylindrical representation, which not only allows pixels at different latitudes to adaptively adopt the best distinct sampling density but also is model-agnostic to most off-the-shelf SR methods, enhancing their performances. Specifically, we start by upsampling each latitude of the input ERP image and design a computationally tractable optimization algorithm to adaptively obtain a (sub)-optimal sampling density for each latitude of the ERP image. Addressing the distortion of ERP, we introduce a new viewport-based training loss based on the original 3D sphere format of the omnidirectional image, which inherently lacks distortion. Finally, we present a simple yet effective recursive progressive omnidirectional SR network to showcase the feasibility of our idea. The experimental results on public datasets demonstrate the effectiveness of the proposed method as well as the consistently superior performance of our method over most state-of-the-art methods both quantitatively and qualitatively.

show abstract

“…Implicit Neural Representation Implicit neural representation [12] approximates continuous signals such as 2D images and 3D shapes. Thanks to the property of INR, various tasks have been proposed including arbitrary image superresolution (SR) [5,20], SR for image warping [19,40,47], view synthesis [28], etc. Among them, local INR [5,19,20] uses both feature maps from a CNN encoder and relative coordinates showing the robustness in generalization to outof-scale datasets.…”

Section: Related Workmentioning

confidence: 99%

“…4, we compare Neural Warping (NW) with LTEW, which is the state-of-the-art method for high-definition warp to verify our method. Following the recent research of arbitraryscale super-resolution for Equirectangular projection (ERP SR), we employ the same evaluation configurations as previous works [6,47]. The measurement of computation time is conducted under a fixed resolution to focus on complexity checking.…”

Section: Ablation Studymentioning

confidence: 99%

Republic of Korea (South Korea)

Kim¹

The Crime of Aggression

View full text Add to dashboard Cite

Despite recent progress in semantic image synthesis, complete control over image style remains a challenging problem. Existing methods require reference images to feed style information into semantic layouts, which indicates that the style is constrained by the given image. In this paper, we propose a model named RUCGAN for user controllable semantic image synthesis, which utilizes a singular color to represent the style of a specific semantic region. The proposed network achieves reference-free semantic image synthesis by injecting color as userdesired styles into each semantic layout, and is able to synthesize semantic images with unusual colors. Extensive experimental results on various challenging datasets show that the proposed method outperforms existing methods, and we further provide an interactive UI to demonstrate the advantage of our approach for style controllability. The codes and UI are available at: https://github.com/BenjaminJonghyun/RUCGAN

show abstract

SphereSR: $360^{\circ}$ Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation

Cited by 32 publications

References 23 publications

Materials Science at Beihang: A Special Issue of Materials Lab Dedicated to the 70th Anniversary of Beihang University

Materials Science at Beihang: A Special Issue of Materials Lab Dedicated to the 70th Anniversary of Beihang University

Spherical Pseudo-Cylindrical Representation for Omnidirectional Image Super-resolution

Republic of Korea (South Korea)

Contact Info

Product

Resources

About