Kernel Foveated Rendering

Meng, Xiaoxu; Du, Ruofei; Zwicker, Matthias; Varshney, Amitabh

doi:10.1145/3203199

Cited by 79 publications

(52 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meng et al . [MDZV18] present a different, analytical approach, where they use log‐polar mappings to parameterize foveated rendering based on the receptor distribution on the retina and thus reducing cost of the rendering step.…”

Section: Related Workmentioning

confidence: 99%

“…The first factor, eccentricity confidence , is based on the falloff in the eye's visual acuity (spacial distinguishability) with larger eccentricity [Adl65], due to the lower receptor density distribution mapped to peripheral vision. This is the property that foveation methods [GFD*12, PSK*16, WRK*16, MDZV18] generally exploit, having especially the cones in mind [PSK*16]. We use

\begin{matrix} true \\ right c_{e} (ϕ) = S_{1} ((), \frac{ϕ - ϕ_{F}}{ϕ_{0} - ϕ_{F}}), \end{matrix}

with

S_{1}

the smooth‐step function,

ϕ

the eccentricity of the current sample,

ϕ_{F}

the angle covered by the fovea and

ϕ_{0}

the angle we expect to be fully into the periphery.…”

Section: Identifying Missing Informationmentioning

confidence: 99%

“…Rendering photo‐realistic images for virtual reality headsets or head‐mounted displays (HMDs) is a demanding task due to the high computational cost and strict frame‐rate constraints [Vla16]. Exploiting weaknesses in the human visual system (HVS) is an established way to loosen constraints and to reduce computation times in rendering [GFD*12, MDZV18, PSK*16, WRK*16]. One of these weaknesses is a less accurate perception in the periphery of the gaze direction; methods targeting this are commonly referred to as foveated rendering techniques.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Time‐Warped Foveated Rendering for Virtual Reality Headsets

Franke

Fink

Martschinke

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

Rendering in real time for virtual reality headsets with high user immersion is challenging due to strict framerate constraints as well as due to a low tolerance for artefacts. Eye tracking‐based foveated rendering presents an opportunity to strongly increase performance without loss of perceived visual quality. To this end, we propose a novel foveated rendering method for virtual reality headsets with integrated eye tracking hardware. Our method comprises recycling pixels in the periphery by spatio‐temporally reprojecting them from previous frames. Artefacts and disocclusions caused by this reprojection are detected and re‐evaluated according to a confidence value that is determined by a newly introduced formalized perception‐based metric, referred to as confidence function. The foveal region, as well as areas with low confidence values, are redrawn efficiently, as the confidence value allows for the delicate regulation of hierarchical geometry and pixel culling. Hence, the average primitive processing and shading costs are lowered dramatically. Evaluated against regular rendering as well as established foveated rendering methods, our approach shows increased performance in both cases. Furthermore, our method is not restricted to static scenes and provides an acceleration structure for post‐processing passes.

show abstract

Section: Related Workmentioning

confidence: 99%

\begin{matrix} true \\ right c_{e} (ϕ) = S_{1} ((), \frac{ϕ - ϕ_{F}}{ϕ_{0} - ϕ_{F}}), \end{matrix}

with

S_{1}

the smooth‐step function,

ϕ

the eccentricity of the current sample,

ϕ_{F}

the angle covered by the fovea and

ϕ_{0}

the angle we expect to be fully into the periphery.…”

Section: Identifying Missing Informationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Time‐Warped Foveated Rendering for Virtual Reality Headsets

Franke

Fink

Martschinke

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Foveated rendering (FR) uses a spatial quality profile that aims to match the visual acuity of the eye [5,12,16,18,19,23]. The scene is divided into multiple regions each of which is rendered according to its angular eccentricity from the gaze point using, e.g., variable rate shading [19].…”

Section: Introductionmentioning

confidence: 99%

On the Interplay of Foveated Rendering and Video Encoding

Illahi

Siekkinen

Kämäräinen

et al. 2020

26th ACM Symposium on Virtual Reality Software and Technology

View full text Add to dashboard Cite

Humans have sharp central vision but low peripheral visual acuity. Prior work has taken advantage of this phenomenon in two ways: foveated rendering (FR) reduces the computational workload of rendering by producing lower visual quality for peripheral regions and foveated video encoding (FVE) reduces the bitrate of streamed video through heavier compression of peripheral regions. Remote rendering systems require both rendering and video encoding and the two techniques can be combined to reduce both computing and bandwidth consumption. We report early results from such a combination with remote VR rendering. The results highlight that FR causes large bitrate overhead when combined with normal video encoding but combining it with FVE can mitigate it.

show abstract

“…Foveation has been used in image/video compression for reduction of the amount of data to be codified, graphic rendering [18][19][20][21], for streaming data through the Internet or other channels [4,5,22] and for stereo disparity estimation [23]. In such application, the foveation method should comply with a pleasant appearance for human visualization purposes, after reconstruction.…”

Section: Introductionmentioning

confidence: 99%

Dynamic multifoveated structure for real-time vision tasks in robotic systems

Medeiros

Gomes

Clua

et al. 2019

J Real-Time Image Proc

View full text Add to dashboard Cite

Foveation is a technique that allows real-time image processing by drastically reducing the amount of visual data without loosing essential information around some focused area. When a robot needs to pay attention at two or more regions of the image at the same time, e.g., for tracking two or more objects, multifoveation is necessary. In this case, computing features twice in the intersections between the different foveated structures, which could linearly increase the processing time, must be avoided. To solve this redundancy removal problem, we propose two algorithms. The first one is based on the previous calculation of redundant blocks and the second one is based on a pixel-by-pixel processing at execution time. Experimental results show a gain in processing time for the block-based model in comparison with the pixel-by-pixel and also of both in comparison with other approaches that sequentially calculate various single foveated images. Robotics vision and other tasks related to dynamic visual attention, as recognition, real-time surveillance, video transmission, and image rendering, are examples of applications that can rely on and strongly benefit from such model.

show abstract

Kernel Foveated Rendering

Cited by 79 publications

References 38 publications

Time‐Warped Foveated Rendering for Virtual Reality Headsets

Time‐Warped Foveated Rendering for Virtual Reality Headsets

On the Interplay of Foveated Rendering and Video Encoding

Dynamic multifoveated structure for real-time vision tasks in robotic systems

Contact Info

Product

Resources

About