Energy-Efficient Video Processing for Virtual Reality

Leng, Yonggang; Huang, Jian; Chen, Chi‐Chun; Sun, Qiuyue; Zhu, Yuhao

doi:10.1109/mm.2020.2985692

Cited by 24 publications

(26 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As the size of a single decoded video frame becomes as large as tens of megabytes for a highresolution video (e.g., 24MB for a 4K video), the data movement to/from DRAM alone contributes more than 30% of the total system energy consumption. State-of-the-art VR streaming schemes [61,108], which significantly reduce the energy for computation compared to traditional schemes [33,40], show similar trends in the energy consumption breakdown. In this paper, we find two inefficiencies of conventional video display schemes that result in the underutilization of advanced architectural features widely available in modern mobile systems.…”

Section: Introductionmentioning

confidence: 91%

“…Most mobile applications follow a sequence of steps that use multiple SoC IPs. In today's software and hardware architectures, an application (which runs on a CPU core) orchestrates each step and interaction with the IPs (e.g., VD, DC, NIC) [61,69,105]. The application interacts with and issues tasks to an IP driver using an application programming interface (API), which serves as an abstraction layer for the hardware.…”

Section: Mobile Soc Micro-architecturementioning

confidence: 99%

“…An I-Type macroblock is reconstructed from its neighboring macroblocks of the same frame, whereas a P-Type and B-Type macroblock is reconstructed from the macroblocks in the previous and previous/later encoded frames, respectively, that are indicated by the extra information stored as metadata (motion vectors). Finally, a decoded macroblock is written to the frame buffer in the host DRAM ( 3 ) [61,105] in preparation for the next stage. Projection.…”

Section: Planar and Vr Video Processingmentioning

confidence: 99%

“…However, each VR frame must go through a set of projective transformation (PT) operations before being displayed. Therefore, each decoded frame is forwarded to the GPU ( a ), which performs PT operations and writes the processed frame back to the DRAM frame buffer ( b ) [61,108]. Displaying.…”

Section: Planar and Vr Video Processingmentioning

confidence: 99%

See 3 more Smart Citations

BurstLink: Techniques for Energy-Efficient Conventional and Virtual Reality Video Display

Haj-Yahya¹,

Park²,

Bera³

et al. 2021

Preprint

View full text Add to dashboard Cite

Conventional planar video streaming is the most popular application in mobile systems, and the rapid growth of 360 • video content and virtual reality (VR) devices are accelerating the adoption of VR video streaming. Unfortunately, video streaming consumes significant system energy due to high power consumption of the system components (e.g., DRAM, display interfaces, and display panel) involved in this process.We propose BurstLink, a novel system-level technique that improves the energy efficiency of planar and VR video streaming. BurstLink is based on two key ideas. First, BurstLink directly transfers a decoded video frame from the host system (i.e., the video decoder or GPU) to the display panel, bypassing the host DRAM. To this end, we extend the display panel with a double remote frame buffer (DRFB), instead of the DRAM's double frame buffer, so that the system can directly update the DRFB with a new frame while updating the panel's pixels with the current frame stored in the DRFB. Second, BurstLink transfers a complete decoded frame to the display panel in a single burst, using the maximum bandwidth of modern display interfaces. Unlike conventional systems where the frame transfer rate is limited by the pixel-update throughput of the display panel, BurstLink can always take full advantage of the high bandwidth of modern display interfaces by decoupling the frame transfer from the pixel update as enabled by the DRFB. This direct and burst frame transfer of BurstLink significantly reduces energy consumption in video display by 1) reducing accesses to the host DRAM, 2) increasing the system's residency at idle power states, and 3) enabling temporal power gating of several system components after quickly transferring each frame into the DRFB.BurstLink can be easily implemented in modern mobile systems with minimal changes to the video display pipeline. We evaluate BurstLink using an analytical power model that we rigorously validate on a real modern mobile system. Our evaluation shows that BurstLink reduces system energy consumption for 4K planar and VR video streaming by 41% and 33%, respectively, and provides an even higher reduction as display resolution and/or display refresh rate increases.

show abstract

Section: Introductionmentioning

confidence: 91%

Section: Mobile Soc Micro-architecturementioning

confidence: 99%

Section: Planar and Vr Video Processingmentioning

confidence: 99%

Section: Planar and Vr Video Processingmentioning

confidence: 99%

See 2 more Smart Citations

BurstLink: Techniques for Energy-Efficient Conventional and Virtual Reality Video Display

Haj-Yahya¹,

Park²,

Bera³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, it is likely to utilize cloud computing for image rendering in the near future, by virtue of cloud VR [40][41][42]. Currently, the display accounts for 7-17% of the total power of a VR device [43][44][45]. However, the display accounts for about 30% of the total power of a cloud VR system.…”

Section: Performance Evaluation For Power-savingmentioning

confidence: 99%

Study on the Effect of Gaze Position and Image Brightness on Peripheral Dimming Technique

Kim

Jeong

et al. 2021

Electronics

View full text Add to dashboard Cite

Here, we study a low-power technique for displays based on gaze tracking, called peripheral dimming. In this work, the threshold levels of the lightness reduction ratio (LRR), where people notice differences in brightness, depending on gaze positions and image brightness, are investigated. A psychophysical experiment with five gaze positions and three image brightness conditions is performed, and the estimated threshold levels are obtained. To investigate the significance of the differences between the threshold levels, the overlap method and the Bayesian estimation (BEST) analysis are performed. The analysis results show that the difference of the threshold levels depending on the conditions is insignificant. Thus, the proposed technique can operate with a constant LRR level, regardless of the gaze position or image brightness, while maintaining the perceptual image quality. In addition, the proposed technique reduces the power consumption of virtual reality (VR) displays by 12–14% on average. We believe that the peripheral dimming technique would contribute to reducing the power of the self-luminous displays used for VR headsets with an integrated eye tracker.

show abstract