Hardware-Friendly Multiple Transform Selection Module for the VVC Standard

Abstract

The H.266/versatile video coding (VVC) standard is the most recent ITU/ISO video coding standard finalized in July 2020. VVC includes several new coding tools at different levels of the coding scheme. These coding tools enable a significant bitrate saving of up to 50% for the same subjective video quality than its predecessor H.265/high efficiency video coding (HEVC). Among these tools, we can cite the multiple transform selection (MTS) which selects at the encoder horizontal and vertical transforms among three trigonometrical transforms, including discrete cosine transform (DCT) type II, discrete sine transform (DST) type VII and DCT type VIII. Unlike the DCT-II, the DST-VII does not have efficient fast algorithmic implementation. Moreover, the MTS increases the memory usage required to store the coefficients of the three transforms. Consequently, this paper targets an efficient approximation of the DST-VII kernel based on the DCT-II and adjustment stage. The approximation of the DST-VII is modeled as an integer optimization problem jointly minimizing the error and the orthogonality of the approximate DST-VII under sparsity constraint of the adjustment stage. The sparse nonlinear optimizer (SNOPT) solver with an additional relaxation constraint is used to solve the problem and find the best sparse adjustment band matrices for different transform sizes. The DCT-VIII is then computed from the approximate DST-VII with pre/post processing operations involving only sign changes and input/output reordering. The proposed approximation provides a significant reduction in both arithmetic operations and memory usage. Moreover, it preserves the coding gain brought by the MTS under the VVC reference software. These advantages make our solution suitable for energy-efficient hardware H.266//VVC encoders and decoders deployed on consumer electronic devices.

Low-Complexity Multiple Transform Selection Combining Multi-Type Tree Partition Algorithm for Versatile Video Coding

“…In [ 39 ], an entropy-based method was proposed to replace the standard rate estimation. In [ 40 ], the approximation of DCT-VII was modelled to reduce the computation. By combining the histogram of oriented gradient features and the depth information, Wang et al.…”

Fast CU Decision Algorithm Based on Texture Complexity and CNN for VVC

“…Compared with HEVC, the QTMT division used in VVC can generate more types of subunits and can better adapt to different video contents. In VVC, there are six ways to classify CU, as shown in Figure 1: Non-split (NS), quadrinomial tree (QT), vertical binary tree (BTV), horizontal binary tree (BTH), vertical trinomial tree (TTV), and horizontal trinomial tree (TTH) [2] . These six division patterns can form a candidate list, and the optimal pattern 𝑛 * is the pattern with the smallest RD Cost in the candidate list 𝑛.…”

“…1 𝑅𝐷 𝐶𝑜𝑠𝑡 = 𝑆𝑆𝐸 + 𝜆 × 𝐵𝑖𝑡 𝑚𝑜𝑑𝑒 (1) 𝑛 * = 𝑛 𝑚𝑖𝑛 𝑅𝐷 𝐶𝑜𝑠𝑡 (2) Where SSE is the distortion of luminance and chrominance, 𝐵𝑖𝑡 𝑚𝑜𝑑𝑒 is the cost of in-frame prediction, and λ is the Lagrangian multiplier. 𝑛 1 contains six division modes, NS, QT, VBT, HBT, VTT, and HTT, and 𝑛 2 contains five division modes, NS, VBT, HBT, VTT, and HTT.…”

Editorial Real-Time Implementation of VVC Standard for Consumer Electronic Devices

“…2) Hardware implementations of VVC codec on FPGA and ASIC devices. The papers accepted for publication in this special issue are briefly described, for more details, the reader may refer to the corresponding paper [5]- [9]. These papers address different aspects to design real time VVC encoders and decoders from tools optimizations [5], [6] for software and hardware VVC decoders, complexity reduction of VVC encoder [7], [9] to the design of real time software VVC encoders [8].…”

“…Additionally in terms of resources needed for the implementations, the use of the three transforms MTS increases the memory usage because it is needed to store more coefficients. Authors in [5] proposed an efficient approximation of the DST-VII based on the DCT-II and adjustment stage. This approximation has been solved as an integer optimization problem.…”