Scalable Omnidirectional Video Coding for Real-Time Virtual Reality Applications

Liu, Deyang; An, Ping; Ma, Ran; Zhan, Wei; Ai, Liefu

doi:10.1109/access.2018.2873367

Cited by 19 publications

(13 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this technique, the motion compensated residue errors are encoded through re-quantization and variable length coding (VLC) [2], [25]. A scalable omni-directional video coding scheme for virtual reality applications is presented in [26]. Xie et al [21] proposed a hybrid domain transcoder, which provides the trade-off between the computational cost and video quality.…”

Section: Literature Reviewmentioning

confidence: 99%

Block Based Enhancement using Deep Learning for Conversion of Low Resolution AVS Video to High Resolution HEVC Video

Qureshi¹,

Nawaz²

2022

Preprint

View full text Add to dashboard Cite

Conversion of one video bitstream to another video bitstream is a challenging task in the heterogeneous transcoder due to different video formats. In this paper, a region of interest (ROI) based super resolution technique is used to convert the lowresolution AVS (audio video standard) video to high definition HEVC (high efficiency video coding) video. Firstly, we classify a low-resolution video frame into small blocks by using visual characteristics, transform coefficients, and motion vector (MV) of a video. These blocks are further classified as blocks of most interest (BOMI), blocks of less interest (BOLI) and blocks of noninterest (BONI). The BONI blocks are considered as background blocks due to less interest in video and remains unchanged during SR process. Secondly, we apply deep learning based super resolution method on low resolution BOMI, and BOLI blocks to enhance the visual quality. The BOMI and BOLI blocks have high attention due to ROI that include some motion and contrast of the objects. The proposed method saves 20% to 30% computational time and obtained appreciable results as compared with full frame based super resolution method. We have tested our method on different official video sequences with resolution of 1K, 2K, and 4K. Our proposed method has an efficient visual performance in contrast to the full frame-based super resolution method.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Block Based Enhancement using Deep Learning for Conversion of Low Resolution AVS Video to High Resolution HEVC Video

Qureshi¹,

Nawaz²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…They were able to reduce the bitrate by approximately 87% at a modest quality degradation in the RoI. A triple-layer scalable video coding scheme was conceived in [10] for improving the coding efficiency of Virtual Reality (VR) applications, where the particular video frame portions outside the FoV were encoded as the Base Layer (BL) only, while the FoV portions were further enhanced by the ELs.…”

Section: State-of-the-artmentioning

confidence: 99%

“…(9) imposes a constraint on the FEC code-rate allocation for having an FEC coded rate below the upper bound of T , where |f l i,j | represents the frame length of the video frame f l i,j in terms of the number of bits. Additionally, (10) represents the constraint that the FEC code-rate has to be selected from the legitimate code-rate set R. (8) can be reformulated as the weighted uncoded source-rate summation of the three classes, as given by:…”

Section: A Problem Formulationmentioning

confidence: 99%

See 1 more Smart Citation

Scalable Panoramic Wireless Video Streaming Relying on Optimal-Rate FEC-Coded Adaptive QAM

Zhang

Huo

et al. 2020

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

The demanding bitrate requirements of supporting flawless streaming of immersive video remains a challenge. To minimize the bandwidth without degrading the user's experience, we may opt for partitioning the panoramic video frame into numerous tiles and only transmit those covered by the predicted Field of View (FoV), but naturally, this philosophy critically hinges on the accuracy of the Head Movement Prediction (HMP). However, since the HMP is never 100% accurate, rebuffering of the missing portions due to misprediction is likely to introduce video freezes or artefacts, which may significantly degrade the users' experience. Hence, instead of only transmitting the FoV tiles, Scalable Video Coding (SVC) comes to rescue. Explicitly, in SVC schemes, multiple layers having different importance provide a promising solution, where the basic quality of the entire panoramic video is supported by the Base Layer (BL) that only requires a low bitrate, while the Enhancement Layers (EL) are invoked for enhancing the quality of the predicted FoV. In this treatise, we propose coding rate adaptation assisted nearinstantaneously Adaptive Quadrature Amplitude Modulation (AQAM) for layered panoramic video streaming. In our design, we categorize the video streaming into three priority classes according to the FoV and the SVC layer index, each of which is mapped to the most appropriate modulation mode determined by the instantaneous channel quality. Furthermore, we conceive an Evolutionary Algorithm (EA) assisted Forward Error Correction (FEC) coding rate optimization method for providing Unequal Error Protection (UEP) in order to maximize the uncoded sourcerate according to the inter-frame and inter-layer decoding dependency for tile-based panoramic video streaming. The simulation results show that the proposed AQAM assisted UEP scheme configured by our EA assisted coding rate optimization algorithm significantly improves the overall video performance compared to its Equal Error Protection (EEP) counterpart, and provides perceptually pleasing video quality across a wide range of channel conditions by selecting the most appropriate modulation mode based on the instantaneous channel Signal-to-Noise Ratio (SNR).

show abstract

“…With the rapid growth of industrial video applications [1] (e.g. video-on-demand (VoD) [2], live over-the-top (OTT) [3], Television broadcasting, video surveillance and conferencing [4], 360 panoramic video [5], etc. ), Ultra high-definition (UHD) video format is expected as one of the most attractive services in 5G network.…”

Section: Introductionmentioning

confidence: 99%

Fine-Grained Region Adaptive Loop Filter for Super-Block Video Coding

et al. 2020

View full text Add to dashboard Cite

In video coding, in-loop filter plays an essential role to reduce compression artifacts. This paper introduces a Fine-grained Region Adaptive Loop Filter (FR-ALF) approach for super-block coding based on the High Efficiency Video Coding (HEVC) framework. To address the over-smoothing and low-efficient problems, we introduce a flexible low-complexity FR-ALF method that achieves a finer granularity to control the filtering effect. Moreover, we design a new communication mechanism in order to avoid encoding the same overhead information. Experimental results show that compared with recent advances, our FR-ALF method achieves a satisfactory bit-rate saving by 2.76% on average, and consumes an affordable complexity increment.INDEX TERMS Adaptive loop filter (ALF), super-block coding, ultra high-definition video coding.

show abstract

Scalable Omnidirectional Video Coding for Real-Time Virtual Reality Applications

Cited by 19 publications

References 16 publications

Block Based Enhancement using Deep Learning for Conversion of Low Resolution AVS Video to High Resolution HEVC Video

Block Based Enhancement using Deep Learning for Conversion of Low Resolution AVS Video to High Resolution HEVC Video

Scalable Panoramic Wireless Video Streaming Relying on Optimal-Rate FEC-Coded Adaptive QAM

Fine-Grained Region Adaptive Loop Filter for Super-Block Video Coding

Contact Info

Product

Resources

About