Stabilizing the Convolution Operations for Neural Network-Based Image and Video Codecs for Machines

Zhang, Honglei; Le, Nam; Cricri, Francesco; Ahonen, Jukka; Tavakoli, Hamed Rezazadegan

doi:10.1109/icmew59549.2023.00036

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2023

2024

Publication Types

Select...

Other1

Article1

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

NN-VVC: Versatile Video Coding boosted by self-supervisedly learned image coding for machines

Ahonen,

Le,

Zhang

et al. 2023

2023 IEEE International Symposium on Multimedia (ISM)

View full text Add to dashboard Cite

NN-VVC: Versatile Video Coding boosted by self-supervisedly learned image coding for machines

Ahonen,

Le,

Zhang

et al. 2023

2023 IEEE International Symposium on Multimedia (ISM)

View full text Add to dashboard Cite

NN-VVC: A Hybrid Learned-Conventional Video Codec Targeting Humans and Machines

Ahonen,

Le,

Zhang

et al. 2024

Int. J. Semantic Computing

View full text Add to dashboard Cite

Advancements in artificial intelligence have significantly increased the use of images and videos in machine analysis algorithms, predominantly neural networks. However, the traditional methods of compressing, storing and transmitting media have been optimized for human viewers rather than machines. Current research in coding images and videos for machine analysis has evolved in two distinct paths. The first is characterized by End-to-End (E2E) learned codes, which show promising results in image coding but have yet to match the performance of leading Conventional Video Codecs (CVC) and suffer from a lack of interoperability. The second path optimizes CVC, such as the Versatile Video Coding (VVC) standard, for machine-oriented reconstruction. Although CVC-based approaches enjoy widespread hardware and software compatibility and interoperability, they often fall short in machine task performance, especially at lower bitrates. This paper proposes a novel hybrid codec for machines named NN-VVC, which combines the advantages of an E2E-learned image codec and a CVC to achieve high performance in both image and video coding for machines. Our experiments show that the proposed system achieved up to −43.20% and −26.8% Bjøntegaard Delta rate reduction over VVC for image and video data, respectively, when evaluated on multiple different datasets and machine vision tasks according to the common test conditions designed by the VCM study group in MPEG standardization activities. Furthermore, to improve reconstruction quality, we introduce a human-focused branch into our codec, enhancing the visual appeal of reconstructions intended for human supervision of the machine-oriented main branch.

show abstract

Stabilizing the Convolution Operations for Neural Network-Based Image and Video Codecs for Machines

Cited by 2 publications

References 10 publications

NN-VVC: Versatile Video Coding boosted by self-supervisedly learned image coding for machines

NN-VVC: Versatile Video Coding boosted by self-supervisedly learned image coding for machines

NN-VVC: A Hybrid Learned-Conventional Video Codec Targeting Humans and Machines

Contact Info

Product

Resources

About