Learning Foveated Reconstruction to Preserve Perceived Image Statistics

Surace, Luca; Wernikowski, Marek; Tursun, Cara; Mantiuk, Radosław; Didyk, Piotr

doi:10.48550/arxiv.2108.03499

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This provided computational support for the idea that the HVS foveated nature may confer a functional advantage for scene representation. Surace et al [147] proposed a procedure to train a generative network for foveated image reconstruction. This procedure penalized perceptually significant deviations in the output to maintain perceived rather than natural image statistics.…”

Section: Muti-spatial Resolution For Image/videomentioning

confidence: 99%

Foveated rendering: A state-of-the-art survey

2023

View full text Add to dashboard Cite

Recently, virtual reality (VR) technology has been widely used in medical, military, manufacturing, entertainment, and other fields. These applications must simulate different complex material surfaces, various dynamic objects, and complex physical phenomena, increasing the complexity of VR scenes. Current computing devices cannot efficiently render these complex scenes in real time, and delayed rendering makes the content observed by the user inconsistent with the user’s interaction, causing discomfort. Foveated rendering is a promising technique that can accelerate rendering. It takes advantage of human eyes’ inherent features and renders different regions with different qualities without sacrificing perceived visual quality. Foveated rendering research has a history of 31 years and is mainly focused on solving the following three problems. The first is to apply perceptual models of the human visual system into foveated rendering. The second is to render the image with different qualities according to foveation principles. The third is to integrate foveated rendering into existing rendering paradigms to improve rendering performance. In this survey, we review foveated rendering research from 1990 to 2021. We first revisit the visual perceptual models related to foveated rendering. Subsequently, we propose a new foveated rendering taxonomy and then classify and review the research on this basis. Finally, we discuss potential opportunities and open questions in the foveated rendering field. We anticipate that this survey will provide new researchers with a high-level overview of the state-of-the-art in this field, furnish experts with up-to-date information, and offer ideas alongside a framework to VR display software and hardware designers and engineers.

show abstract

Section: Muti-spatial Resolution For Image/videomentioning

confidence: 99%

Foveated rendering: A state-of-the-art survey

2023

View full text Add to dashboard Cite

show abstract

“…In the literature, there have been a lot of studies on foveated contents (i.e., images or videos) [3], [20], [12], [27], [28], [4], [29], [7], and [30]. Among them, there are, however, only some on 360 • contents [3], [28], [4], [29], and [7].…”

Section: A Foveated 360 • Content Subjective Quality Assessmentmentioning

confidence: 99%

An Effective Foveated 360° Image Assessment Based on Graph Convolution Network

Hương

Tran

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Virtual reality (VR) has been adopted in various fields such as entertainment, education, healthcare, and the military, due to its ability to provide an immersive experience to users. However, 360 • images, one of the main components in VR systems, have bulky sizes and thus require effective transmitting and rendering solutions. One of the potential solutions is to use foveated technologies, that take advantage of the foveation feature of the human eyes. Foveated technologies can significantly reduce the data required for transmission and computation complexity in rendering. However, understanding the impact of foveated 360 • images on human quality perception is still limited. This paper addresses the above problems by proposing an accurate machine-learning-based quality assessment model for foveated 360 • images. The proposed model is proven to outperform the three cutting-edge machine-learning-based models, which apply deep learning techniques and 25 traditional-metric-based models (or analytical-function-based-models), which utilize analytical functions. It is also expected that our model helps to evaluate and improve 360 • content streaming and rendering solutions to further reduce data sizes while ensuring user experience. Also, this model could be used as a building block to construct quality assessment methods for 360 • videos, that are reserved for our future work. The source code is available at https://github.com/telagment/FoVGCN.

show abstract

Learning Foveated Reconstruction to Preserve Perceived Image Statistics

Cited by 2 publications

References 0 publications

Foveated rendering: A state-of-the-art survey

Foveated rendering: A state-of-the-art survey

An Effective Foveated 360° Image Assessment Based on Graph Convolution Network

Contact Info

Product

Resources

About