Learning-Based Video Coding with Joint Deep Compression and Enhancement

Zhao, Tiesong; Feng, Weize; Zeng, Hongji; Xu, Yiwen; Niu, Yuzhen; Liu, Jiaying

doi:10.1145/3503161.3548314

Cited by 12 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides, Lu et al further presented an online encoder updating scheme [40] from the perspective of content adaptation and error propagation. Afterwards, a series of end-to-end video compression algorithms [41], [42], [43], [44], [45], [46], [47], [48] were put forward to improve the coding performance. More specifically, Hu et al [41] proposed an end-to-end video compression framework by converting the input video to the latent code representation.…”

Section: Learning-based Video Codingmentioning

confidence: 99%

“…More specifically, Hu et al [41] proposed an end-to-end video compression framework by converting the input video to the latent code representation. The relevant techniques of recurrent learning [42] and adversarial learning [45], [47] were also introduced in the end-to-end compression framework. These learning-based compression algorithms could achieve promising coding efficiency in universal scenes, while there is still room for improvement considering the specific application scenarios such as talking face videos.…”

Section: Learning-based Video Codingmentioning

confidence: 99%

See 1 more Smart Citation

Beyond Keypoint Coding: Temporal Evolution Inference with Compact Feature Representation for Talking Face Video Compression

Chen

Wang

et al. 2022

2022 Data Compression Conference (DCC)

View full text Add to dashboard Cite

In this paper, we propose a novel framework for Interactive Face Video Coding (IFVC), which allows humans to interact with the intrinsic visual representations instead of the signals. The proposed solution enjoys several distinct advantages, including ultra-compact representation, low delay interaction, and vivid expression and headpose animation. In particular, we propose the Internal Dimension Increase (IDI) based representation, greatly enhancing the fidelity and flexibility in rendering the appearance while maintaining reasonable representation cost. By leveraging strong statistical regularities, the visual signals can be effectively projected into controllable semantics in the three dimensional space (e.g., mouth motion, eye blinking, head rotation and head translation), which are compressed and transmitted. The editable bitstream, which naturally supports the interactivity at the semantic level, can synthesize the face frames via the strong inference ability of the deep generative model. Experimental results have demonstrated the performance superiority and application prospects of our proposed IFVC scheme. In particular, the proposed scheme not only outperforms the stateof-the-art video coding standard Versatile Video Coding (VVC) and the latest generative compression schemes in terms of rate-distortion performance for face videos, but also enables the interactive coding without introducing additional manipulation processes. Furthermore, the proposed framework is expected to shed lights on the future design of the digital human communication in the metaverse. The project page can be found at https://github.com/Berlin0610/Interactive Face Video Coding.

show abstract