Foveated video streaming for cloud gaming

Illahi, Gazi; Siekkinen, Matti; Masala, Enrico

doi:10.1109/mmsp.2017.8122235

Cited by 16 publications

(10 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The approach works with any off-the-shelf game and requires no modifications to the game engine. In [Illahi et al 2017], we make optimistic but cautious conclusions about the feasibility of FVE in cloud gaming and the potential of bandwidth savings without affecting QoE. In this work, we extend our previous work by validating our observations on feasibility of FVE and potential bandwidth savings with a user study.…”

Section: Cloud Gaming and Fvesupporting

confidence: 53%

See 1 more Smart Citation

Cloud Gaming with Foveated Video Encoding

Illahi

Gemert

Siekkinen

et al. 2020

ACM Trans. Multimedia Comput. Commun. Appl.

Self Cite

View full text Add to dashboard Cite

Cloud gaming enables playing high end games, originally designed for PC or game console setups, on low end devices, such as net-books and smartphones, by offloading graphics rendering to GPU powered cloud servers. However, transmitting the high end graphics requires a large amount of available network bandwidth, even though it is a compressed video stream. Foveated video encoding (FVE) reduces the bandwidth requirement by taking advantage of the non-uniform acuity of human visual system and by knowing where the user is looking. We have designed and implemented a system for cloud gaming with foveated graphics using a consumer grade real-time eye tracker and an open source cloud gaming platform. In this article, we describe the system and its evaluation through measurements with representative games from different genres to understand the effect of parameterization of the FVE scheme on bandwidth requirements and to understand its feasibility from the latency perspective. We also present results from a user study. The results suggest that it is possible to find a "sweet spot" for the encoding parameters so that the users hardly notice the presence of foveated encoding but at the same time the scheme yields most of the bandwidth savings achievable.

show abstract

Section: Cloud Gaming and Fvesupporting

confidence: 53%

“…In our previous work [Illahi et al 2017], we develop a foveated video streaming solution for cloud gaming, wherein…”

Section: Cloud Gaming and Fvementioning

confidence: 99%

Cloud Gaming with Foveated Video Encoding

Illahi

Gemert

Siekkinen

et al. 2020

ACM Trans. Multimedia Comput. Commun. Appl.

Self Cite

View full text Add to dashboard Cite

show abstract

“…For each macroblock the quantization parameter is optimized based on this model. Illahi et al [15] adapted the video encoder of a cloud-gaming application so that it changes the quality of the encoding based on the gaze direction of the player. They adjust the quality parameter of each macroblock based on the current gaze position, increasing the quality in macroblocks the player looks at and decreasing the quality in the remaining ones.…”

Section: Related Workmentioning

confidence: 99%

Foveated Encoding for Large High-Resolution Displays

Frieß

Braun

Bruder

et al. 2021

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Collaborative exploration of scientific data sets across large high-resolution displays requires both high visual detail as well as low-latency transfer of image data (oftentimes inducing the need to trade one for the other). In this work, we present a system that dynamically adapts the encoding quality in such systems in a way that reduces the required bandwidth without impacting the details perceived by one or more observers. Humans perceive sharp, colourful details, in the small foveal region around the centre of the field of view, while information in the periphery is perceived blurred and colourless. We account for this by tracking the gaze of observers, and respectively adapting the quality parameter of each macroblock used by the H.264 encoder, considering the so-called visual acuity fall-off. This allows to substantially reduce the required bandwidth with barely noticeable changes in visual quality, which is crucial for collaborative analysis across display walls at different locations. We demonstrate the reduced overall required bandwidth and the high quality inside the foveated regions using particle rendering and parallel coordinates.

show abstract

“…Typically, rendering algorithms reduce the complexity and decimate some parts of the virtual scene according to the distance to the focus point [5][6][7][8][9]. Besides, some researchers proposed to use similar strategies to set the quantization parameter of the codec, i.e., the foveated video coding [10][11][12], to reduce the bandwidth consumption. In the foveated video streaming, the video quality may change spatially within a frame: encoding at the highest quality where the user gaze is fixated or predicted to fixate and at lower quality elsewhere.…”

Section: Introductionmentioning

confidence: 99%

“…In the foveated video streaming, the video quality may change spatially within a frame: encoding at the highest quality where the user gaze is fixated or predicted to fixate and at lower quality elsewhere. For instance, Illahi et al present a foveated video streaming system for cloud gaming [10,11] that adapts video stream quality by adjusting the encoding parameters on the fly to match the player's gaze position. Evaluation results suggest that such foveated streaming can reduce bandwidth requirements by even more than 50%.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Quality Of Experience For Cloud Virtual Reality Gaming: An Object-Aware Video Encoding

Zou

Feng

Mao

et al. 2021

2021 IEEE International Conference on Multimedia &Amp; Expo Workshops (ICMEW)

View full text Add to dashboard Cite

Cloud gaming is an ideal solution to provide the desired GPU rendering for the Virtual Reality (VR) gaming while reducing the requirements for the weight, price, and capabilities of the head-mounted display. However, transmitting the highresolution video requires a large amount of network bandwidth. In this paper, we present a cloud VR gaming system that significantly improves the coding efficiency by using a proposed Object-Aware Video Encoding (OAVE). Different from the video service, the game engine can offer information about the objects in the game scene. By combining this information and the eye-tracking data, the encoder compressed the rendered video in an object-dependent way that sets the quantization parameter according to the object of the user's gaze. Through the subjective experiment conducted on the prototype of the system, above 40% to 47% bandwidth saving can be achieved compared with the conventional VR gaming system without the proposed optimization.

show abstract

Foveated video streaming for cloud gaming

Cited by 16 publications

References 12 publications

Cloud Gaming with Foveated Video Encoding

Cloud Gaming with Foveated Video Encoding

Foveated Encoding for Large High-Resolution Displays

Enhancing Quality Of Experience For Cloud Virtual Reality Gaming: An Object-Aware Video Encoding

Contact Info

Product

Resources

About