A Rate Control Algorithm for HEVC Considering Visual Saliency

Wei, Henglu; Zhou, Wei; Bai, Rui; Duan, Zhemin

doi:10.23919/apsipa.2018.8659729

Cited by 8 publications

(7 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [20], ROI information presented in the form of spatial and temporal saliency map is first combined and a saliency value for each CU is calculated and used to influence the QP assignment and RDO process. Similar in [19] and [21], saliency value is generated for each CTU, and QP calculation is adjusted to allocate more coding bits to more salient CTUs. To carry out perception-based coding to UHD videos, we choose to formulate a deep neural network to detect human faces from video frames and simultaneously locate the facial landmarks.…”

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“…Dynamic models such as [16] utilize temporal cues to model dynamic saliency, and can be used on videos. Methods such as [19,20] and [21] use the extracted saliency maps to modulate the rate-distortion optimization or quantization process in standard video coding methods to allocate more bits on salient areas of video frames.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Real‐time face perception based encoding strategy optimization method for UHD videos

Wang

Han³

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

Face regions containing rich semantic information appear frequently in the videos. As the video resolution increase dramatically, the face regions will inevitably attract more attentions. This paper proposes a face perception based coding scheme to improve the visual quality of the face regions in UHD videos. A specially tailored face perception model is first utilized to precisely and quickly locate the face regions. Then, a face perception map is generated based on a hierarchical mapping algorithm. Finally, the face perception map is employed as a guidance to optimize the encoding process, including mode decision, block partition and bit allocation. The proposed method is implemented on HEVC to demonstrate the effectiveness. Experimental results on a set of 4K test sequences show that the proposed method can obviously improve the objective and subjective quality of the face regions, while causing only slight quality decline over the rest of the frame. Additionally, the computation required for mode decision and block partition is reduced, thereby saving encoding time cost.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Real‐time face perception based encoding strategy optimization method for UHD videos

Wang

Han³

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

show abstract

“…The authors of [8] allocate the Lidar bitrate budget according to ROI and non-ROI, significantly improving the quality of selfdriving car depth maps. In [37], the encoder allocates the CTU bits according to their visual saliency. Then, the 𝑄𝑃 for each CTU is calculated by the Lagrange multipliers.…”

Section: Gmentioning

confidence: 99%

Content adaptive video compression for autonomous vehicle remote driving

Dror

Birman

Solomon

et al. 2021

Applications of Digital Image Processing XLIV

View full text Add to dashboard Cite

Autonomous vehicles are a promising solution to traffic congestion, air pollution, accidents, and wasted time and resources. However, remote driver intervention may be necessary for extreme situations to ensure safe roadside parking or complete remote takeover. In such cases, high-quality real-time video streaming is crucial for practical remote driving. In a preliminary study, we already presented a region of interest (ROI) HEVC data compression where the image was segmented into two categories of ROI and background, allocating more bandwidth to the ROI, yielding an improvement in the visibility of the classes that essential for driving while transmitting the background with lesser quality. However, migrating bandwidth to the large ROI portion of the image doesn't substantially improve the quality of traffic signs and lights. This work categorized the ROIs into either background, weak ROI, or strong ROI. The simulation-based approach uses a photo-realistic driving scenario database created with the Cognata self-driving car simulation platform. We use semantic segmentation to categorize the compression quality of a Coding Tree Unit (CTU) according to each pixel class. A background CTU can contain only sky, trees, vegetation, or building classes. Essentials for remote driving include significant classes such as roads, road marks, cars, and pedestrians. And most importantly, traffic signs and traffic lights. We apply thresholds to decide if the number of pixels in a CTU of a particular category is enough to declare it as belonging to the strong or weak ROI. Then, we allocate the bandwidth according to the CTU categories. Our results show that the perceptual quality of traffic signs, especially textual signs and traffic lights, improves significantly by up to 5.5 dB compared to the only background and foreground partition, while the weak ROI classes at least retain their original quality.

show abstract

“…A gradient based R-λ model for the intra frame rate control was proposed by Wang et al [19]. In addition, a bit allocation algorithm based on saliency was also proposed in [20]. Sun et al [21] proposed a new perception-based intra frame coding optimization algorithm that adaptively updated bit allocation for the salient and non-salient regions.…”

Section: Introductionmentioning

confidence: 99%

Perception-Based CTU Level Bit Allocation for Intra High Efficiency Video Coding

et al. 2019

View full text Add to dashboard Cite

The conventional rate control algorithm does not fully consider the characteristics of the Human Visual System (HVS). To address this problem, in this paper, a perception based Coding Tree Unit (CTU) level bit allocation algorithm for intra High Efficiency Video Coding (HEVC) is proposed, which aims to minimize the perceptual distortion of each CTU under a given bit rate constraint. Firstly, to better represent perceptual distortion, Perceptually Weighted Mean Squared Error (PWMSE) is adopted instead of Mean Squared Error (MSE). Then, the relationship between Rate (R) and Perceptually Weighted Distortion (D p) is formulated by the proposed R-D p model. Finally, the perceptual weighting factor derived from the R-D p model is used to guide CTU level bit allocation. Experimental results show that the proposed algorithm achieves 6.27% bit rate reduction and 0.38 dB Bjøntegaard Delta Perceptually Weighted Peak Signal to Noise Ratio (BD-PWPSNR) gain on average. Under other seven quality evaluation metrics, the proposed algorithm achieves from 1.57% to 8.95% average bit rate reduction while maintaining the perceptual quality, which significantly outperforms the benchmark schemes. Moreover, the proposed perception-based CTU level bit allocation algorithm also maintains a high rate control accuracy, which reaches 99.997%. INDEX TERMS Perceptual coding, bit allocation, rate control, high efficiency video coding.

show abstract

A Rate Control Algorithm for HEVC Considering Visual Saliency

Cited by 8 publications

References 22 publications

Real‐time face perception based encoding strategy optimization method for UHD videos

Real‐time face perception based encoding strategy optimization method for UHD videos

Content adaptive video compression for autonomous vehicle remote driving

Perception-Based CTU Level Bit Allocation for Intra High Efficiency Video Coding

Contact Info

Product

Resources

About