Saliency-driven image coding improves overall perceived JPEG quality

Abstract

Abstract-Saliency-driven image coding is well worth pursuing. Previous studies on JPEG and JPEG2000 have suggested that region-of-interest coding brings little overall benefit compared to the standard implementation. We show that our saliencydriven variable quantization JPEG coding method significantly improves perceived image quality. To validate our findings, we performed large crowdsourcing experiments involving several hundred contributors, on 44 representative images. To quantify the level of improvement, we devised an approach to equate Likert-type opinions to bitrate differences. Our saliency-driven coding showed 11% bpp average benefit over the standard JPEG. I. INTRODUCTION AND MOTIVATIONRegion-of-interest (ROI) based image compression techniques propose to compress the background more than the foreground in order to improve the perceived image quality. In his studies on two-level ROI based JPEG2000 image coding Bradley [1] has shown that his strategy did not improve on the standard JPEG2000 overall. It only did so at very low bitrates. Harding et al.[2] proposed a binary "visual interest"-guided JPEG2000 compression technique that increased image quality as measured objectively, but did not account for the fact that images of the same bitrate should be compared. Furthermore, in a recent study on perceptual quality in images Alers et al.[3] have shown that image foreground regions are much more important than the background. In view of these results we reconsider ROI based image compression. To better understand the importance of the ROI in perceived quality, we designed our own saliency-driven variable coding strategy.Due to its simplicity and popularity we decided to base our variable quantization technique on JPEG Part 3 rather than working with JPEG2000. One of the intended purposes of variable quantization is "the ability to use the masking properties of the human visual system more effectively, and thereby achieving greater compression rates for the same subjective quality", see [4]. Variable quantization has already been shown to produce better results than the standard JPEG for special applications. For instance, Konstantinides et al. [5] have adjusted the quantization scaling factors in composite documents. Memon et al. [6] used a measure of block activity and type. None of these works evaluate results in terms of perceptual improvement (user studies). Harding et al. [2] have performed limited subjective studies, however due the low numbers of participants their results are inconclusive. Yu et al. [7] perform subjective evaluation as well using a sequential paired comparison quality assessment methodology. Their results are overall not in favor of their encoding technique.

A simple method to obtain visual attention data in head mounted virtual reality

“…This information can be further used to adapt compression parameters when adaptive coding is applied. An example of such an adaptive coding method has been proposed in [19] for conventional images.…”

Foveated Video Coding for Real-Time Streaming Applications

“…Wang et al published an approach to a region of interest (ROI) coding in the context of embedded zerotree wavelet coding [7]. This rnethodology has seen successful applications for still irnages already [8]. When reviewing older contributions, o ne has to carefully distinguish between ROI coding and foveation, as the nomenclature is partially overlapping and ambiguous.…”

Important Region Estimation Using Image Captioning

“…For example, a video can be summarized by detecting key frames [5]. Using the region of interest (ROI) encoding introduced in JPEG2000 [6], images can be stored or transmitted efficiently by encoding the necessary parts with high definition and the remaining parts at a high compression rate [7], [8].…”