Generating High-Quality Panorama by View Synthesis Based on Optical Flow Estimation

Zhang, Wenxin; Wang, Yumei; Liu, Yu

doi:10.3390/s22020470

Cited by 5 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In computer graphics, view synthesis techniques, including geometry and layout prediction, optical flow, depth, and illumination estimation (Song and Funkhouser 2019;Xu et al 2021;Zhang, Wang, and Liu 2022;Wang et al 2022;Somanath and Kurz 2021), are often studied. Spherical structure and texture are modeled using cube maps (Han and Suh 2020), cylinder convolution (Liao et al 2022), predicted panoramic three-dimensional structures (Song et al 2018), and scene symmetry (Hara, Mukuta, and Harada 2021).Most previous generative models are limited in handling fixed scenes and low-resolution images.…”

Section: Related Workmentioning

confidence: 99%

HORIZON: High-Resolution Semantically Controlled Panorama Synthesis

Yan,

Ji,

et al. 2024

AAAI

View full text Add to dashboard Cite

Panorama synthesis endeavors to craft captivating 360-degree visual landscapes, immersing users in the heart of virtual worlds. Nevertheless, contemporary panoramic synthesis techniques grapple with the challenge of semantically guiding the content generation process. Although recent breakthroughs in visual synthesis have unlocked the potential for semantic control in 2D flat images, a direct application of these methods to panorama synthesis yields distorted content. In this study, we unveil an innovative framework for generating high-resolution panoramas, adeptly addressing the issues of spherical distortion and edge discontinuity through sophisticated spherical modeling. Our pioneering approach empowers users with semantic control, harnessing both image and text inputs, while concurrently streamlining the generation of high-resolution panoramas using parallel decoding. We rigorously evaluate our methodology on a diverse array of indoor and outdoor datasets, establishing its superiority over recent related work, in terms of both quantitative and qualitative performance metrics. Our research elevates the controllability, efficiency, and fidelity of panorama synthesis to new levels.

show abstract

Section: Related Workmentioning

confidence: 99%

HORIZON: High-Resolution Semantically Controlled Panorama Synthesis

Yan,

Ji,

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…10, the energy function can only handle linear structures, which may result in straight lines appearing bent in panoramas with ERP format. To address this limitation, [20] introduced a line-preserving energy term to the geodesic-preserving energy term. The line-preserving energy term improves the accuracy of the final panorama by preserving the straightness of lines in the input images during the warping process.…”

Section: Local and Global Warping Rectifyingmentioning

confidence: 99%

Optimizing Image Rectangular Boundaries with Precision: A Genetic Algorithm-Based Approach with Deep Stitching

2023

IJIES

View full text Add to dashboard Cite

This paper presents a novel method for optimizing rectangular boundaries in stitched images using a combination of deep learning and genetic algorithms. The proposed method utilizes an enhanced structure of convolutional neural network (CNN) in order to extract features for residual regression and a genetic algorithm to tune CNN parameters. The performance of the proposed method was evaluated using various metrics, including mean squared error, peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and Fréchet inception distance (FID). The outcomes from the conducted experiment demonstrated that the suggested method achieved high accuracy in rectangular boundary optimization, even for images with complex shapes and orientations. The method was also tested with different parameters, and results indicated that increasing the number of generations and population size led to improved performance. Also, the combination of deep learning algorithm specifically (CNN) and optimization algorithm specifically (genetic algorithm (GA)) led to an increase in the accuracy of the rectified images where the average PSNR reached 23.98, SSIM is 0.8066, and FID is 18.72. The proposed method has potential applications in various areas of study, such as image cropping, object detection, and image segmentation.

show abstract

Generating a full spherical view by modeling the relation between two fisheye images

Flores,

Valiente,

Peidró

et al. 2024

Vis Comput

View full text Add to dashboard Cite

Full spherical views provide advantages in many applications that use visual information. Dual back-to-back fisheye cameras are receiving much attention to obtain this type of view. However, obtaining a high-quality full spherical view is very challenging. In this paper, we propose a correction step that models the relation between the pixels of the pair of fisheye images in polar coordinates. This correction is implemented during the mapping from the unit sphere to the fisheye image using the equidistant fisheye projection. The objective is that the projections of the same point in the pair of images have the same position on the unit sphere after the correction. In this way, they will also have the same position on the equirectangular coordinate system. Consequently, the discontinuity between the spherical views for blending is minimized. Throughout the manuscript, we show that the angular polar coordinates of the same scene point in the fisheye images are related by a sine function and the radial distance coordinates by a linear function. Also, we propose employing a polynomial as a geometric transformation between the pair of spherical views during the image alignment since the relationship between the matching points of pairs of spherical views is not linear, especially in the top/bottom regions. Quantitative evaluations demonstrate that using the correction step improves the quality of the full spherical view, i.e. IQ MS-SSIM, up to 7%. Similarly, using a polynomial improves the IQ MS-SSIM up to 6.29% with respect to using an affine matrix.

show abstract

Generating High-Quality Panorama by View Synthesis Based on Optical Flow Estimation

Cited by 5 publications

References 25 publications

HORIZON: High-Resolution Semantically Controlled Panorama Synthesis

HORIZON: High-Resolution Semantically Controlled Panorama Synthesis

Optimizing Image Rectangular Boundaries with Precision: A Genetic Algorithm-Based Approach with Deep Stitching

Generating a full spherical view by modeling the relation between two fisheye images

Contact Info

Product

Resources

About