CU Partition Mode Decision for HEVC Hardwired Intra Encoder Using Convolution Neural Network

Abstract

The intensive computation of High Efficiency Video Coding (HEVC) engenders challenges for the hardwired encoder in terms of the hardware overhead and the power dissipation. On the other hand, the constrains in hardwired encoder design seriously degrade the efficiency of software oriented fast coding unit (CU) partition mode decision algorithms. A fast algorithm is attributed as VLSI friendly, when it possesses the following properties. First, the maximum complexity of encoding a coding tree unit (CTU) could be reduced. Second, the parallelism of the hardwired encoder should not be deteriorated. Third, the process engine of the fast algorithm must be of low hardware- and power-overhead. In this paper, we devise the convolution neural network based fast algorithm to decrease no less than two CU partition modes in each CTU for full rate-distortion optimization (RDO) processing, thereby reducing the encoder's hardware complexity. As our algorithm does not depend on the correlations among CU depths or spatially nearby CUs, it is friendly to the parallel processing and does not deteriorate the rhythm of RDO pipelining. Experiments illustrated that, an averaged 61.1% intraencoding time was saved, whereas the Bjøntegaard-Delta bit-rate augment is 2.67%. Capitalizing on the optimal arithmetic representation, we developed the high-speed [714 MHz in the worst conditions (125 °C, 0.9 V)] and low-cost (42.5k gate) accelerator for our fast algorithm by using TSMC 65-nm CMOS technology. One accelerator could support HD1080p at 55 frames/s real-time encoding. The corresponding power dissipation was 16.2 mW at 714 MHz. Finally, our accelerator is provided with good scalability. Four accelerators fulfill the throughput requirements of UltraHD-4K at 55 frames/s.

DVC: An End-To-End Deep Video Compression Framework

“…Although they provide highly efficient compression performance, they are manually designed and cannot be jointly optimized in an end-to-end way. For the video compression task, a lot of DNN based methods have been proposed for intra prediction and residual coding [13], mode decision [22], entropy coding [30], post-processing [23]. These methods are used to improve the performance of one particular module of the traditional video compression algorithms instead of building an endto-end compression scheme.…”

CNN-based transform index prediction in multiple transforms framework to assist entropy coding

“…in [10] proposes to use CNNs to provide a binary decision on whether to split the block or not by taking into consideration several features extracted from the block. Similar works have been carried in [11], [12] where a CNN is used to make a-posteriori decision of splitting of coding unit (CU) and prediction unit (PU) blocks. Laude et.…”

Recent Advances on HEVC Inter-Frame Coding: From Optimization to Implementation and Beyond

“…The final version of record is available at http://dx.doi.org/10.1109/TCSVT.2019.2954474 AV1 [141] ---3.0 --Note: 1 LL/RL/LH/RH denote low-delay low complexity, random-access low complexity, low-delay high-efficiency, and random-access high-efficiency configurations in early HM test conditions. [190][191][192][193], block up-sampling for intra frame coding [194], intra mode decision [195][196][197][198][199], transform [200], rate control [201,202], in-loop filtering/post-processing [203][204][205], arithmetic coding [206], or decoder-end artifact-removal and quality enhancement [207,208].…”