Optical Flow and Mode Selection for Learning-based Video Coding

Ladune, Théo; Philippe, Pierrick; Hamidouche, Wassim; Zhang, Lu; Déforges, Olivier

doi:10.1109/mmsp48831.2020.9287049

Cited by 31 publications

(23 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This allows to perform operations adapted to different video contents and leads to compelling performances. Following this idea, AIVC features an additional coding mode called Skip mode [13].…”

Section: Content Adaptation With Skip Modementioning

confidence: 99%

See 1 more Smart Citation

AIVC: Artificial Intelligence based Video Codec

Ladune¹,

Philippe²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper introduces AIVC, an end-to-end neural video codec. It is based on two conditional autoencoders MNet and CNet, for motion compensation and coding. AIVC learns to compress videos using any coding configurations through a single end-to-end rate-distortion optimization. Furthermore, it offers performance competitive with the recent video coder HEVC under several established test conditions. A comprehensive ablation study is performed to evaluate the benefits of the different modules composing AIVC. The implementation is made available at https: //orange-opensource.github.io/AIVC/.

show abstract

Section: Content Adaptation With Skip Modementioning

confidence: 99%

“…Consequently, neural coders performance is not accurately evaluated. This paper introduces AIVC, an AI-based Video Codec featuring both the conditional coding [9] and Skip [13] mechanisms. AIVC is designed to be a versatile codec, able to implement any desired coding configurations.…”

Section: Introduction and Related Workmentioning

confidence: 99%

AIVC: Artificial Intelligence based Video Codec

Ladune¹,

Philippe²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The resulting latent representation is coded and transmitted over the channel. The decoder uses the compressed latent representation Difference Coder Conditional Coder from [11] and [12] Figure 1. Left: Conventional residual coder.…”

Section: Related Workmentioning

confidence: 99%

“…Left: Conventional residual coder. Right: Schematic network structure of conditional autoencoder as proposed as Codec-Net in [11] and [12]. Dashed lines denote transmission over a channel.…”

Section: Related Workmentioning

confidence: 99%

Generalized Difference Coder: A Novel Conditional Autoencoder Structure for Video Compression

Brand¹,

Seiler²,

Schober³

2021

Preprint

View full text Add to dashboard Cite

Motion compensated inter prediction is a common component of all video coders. The concept was established in traditional hybrid coding and successfully transferred to learning-based video compression. To compress the residual signal after prediction, usually the difference of the two signals is compressed using a standard autoencoder. However, information theory tells us that a general conditional coder is more efficient. In this paper, we provide a solid foundation based on information theory and Shannon entropy to show the potentials but also the limits of conditional coding. Building on those results, we then propose the generalized difference coder, a special case of a conditional coder designed to avoid limiting bottlenecks. With this coder, we are able to achieve average rate savings of 27.8% compared to a standard autoencoder, by only adding a moderate complexity overhead of less than 7%.

show abstract

“…In another recent publication, Ladune et al propose ModeNet [11] which jointly trains a motion estimation and transmission and a residual coder in form of a conditional autoencoder. Here a novel concept is employed that the motion network also outputs a fidelity measure of the prediction which is used to derive the equivalent of a skip mode.…”

Section: Prior Artmentioning

confidence: 99%

Intra To Inter: Towards Intra Prediction for Learning-Based Video Coders Using Optical Flow

Brand

Seiler

Kaup

2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Traditional video coders often rely on a block structure for transmission. Here each block is coded separately and sequentially and for each block the encoder can decide whether to use intra or inter prediction. This way, inter and intra prediction can be mixed within a single frame. This has advantages when new areas are uncovered, which were not present in the reference frame, and can hence not be predicted well. These areas are typically predicted using intra prediction. Currently much research goes into end-to-end-trained video coders which do not operate on a block level and typically use dense motion fields for inter prediction. There it is more difficult to incorporate intra prediction for uncovered regions. In this paper we propose a novel concept which enables us to reinterpret classical angular intra prediction in a way that we can transmit it as part of the dense motion field. We can save an average of 18% rate for the transmission of the motion vectors for the same quality of the prediction image.

show abstract

Optical Flow and Mode Selection for Learning-based Video Coding

Cited by 31 publications

References 10 publications

AIVC: Artificial Intelligence based Video Codec

AIVC: Artificial Intelligence based Video Codec

Generalized Difference Coder: A Novel Conditional Autoencoder Structure for Video Compression

Intra To Inter: Towards Intra Prediction for Learning-Based Video Coders Using Optical Flow

Contact Info

Product

Resources

About