Content-adaptive convolutional neural network post-processing filter

Santamaría, María Paz Diago; Lam, Yat-Hong; Cricri, Francesco; Lainema, Jani; Ghaznavi-Youvalari, Ramin; Zhang, Honglei; Hannuksela, Miska M.; Rahtu, Esa; Gaubbuj, Moncef

doi:10.1109/ism52913.2021.00025

Cited by 12 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An offline pre-trained CNN is over-fitted at test time using the target video sequence as input data. Following [9,10], only the bias terms are considered, aiming to decrease the number of weights to update.…”

Section: Over-fittingmentioning

confidence: 99%

“…The filter to be used is selected by means of Rate-Distortion Optimisation (RDO) at the encoder side and the associated index is signalled within the video bitstream. A few other strategies addressed content adaptation through NN over-fitting [8][9][10], with the weight-update being sent to the decoder side along the video bitstream. Lam et al [8,9] studied encoder side over-fitting to enable content adaptation for a pre-trained postprocessing CNN filter.…”

Section: Introductionmentioning

confidence: 99%

“…The weight-update is signalled along the video bitstream to reconstruct the over-fitted CNN at the decoder side, prior to the filtering process. Santamaria et al [10] expanded this approach by adding syntax elements to the video bitstream to indicate whether the post-processing filter is to be used on a Coding Tree Unit (CTU). Furthermore, the Lempel-Ziv-Markov chain Algorithm 2 (LZMA2), which is used by the 7z format [11], was used to compress the weight-update.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Content-Adaptive Neural Network Post-Processing Filter with NNR-Coded Weight-Updates

Santamaría

Cricri

Lainema

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Neural Network (NN) filters improve the perceptual quality of reconstructed videos by reducing compression artefacts. For content adaptation, a few NN-filters use over-fitting. As the adaptation signal is a weight-update, compression is required to minimise significant bitrate overheads. Most approaches, however, use generic data compression algorithms, which are inadequate for coding NN weight-updates. This work introduces a content-adaptive NN post-processing filter with weight-updates coded using the Neural Network compression and Representation (NNR) standard. The bitrate overhead is further decreased by over-fitting only a subset of weights, selected via energy-based analysis. The proposed filter saved about 4.57% (Y), 10.33% (Cb), 6.53% (Cr) Bjøntegaard Delta rate (BD-rate) on top of the Versatile Video Coding (VVC) Test Model (VTM) 11.0 with NN-based Video Coding (NNVC) 1.0, in Random Access (RA) configuration. Compared to the non-over-fitted NN, the performance was doubled; and compared to 7z, NNR reduced the bitrate of the weight-update by ∼64%.

show abstract

Section: Over-fittingmentioning

confidence: 99%