Observer Dependent Lossy Image Compression

Abstract

Deep neural networks have recently advanced the state-of-the-art in image compression and surpassed many traditional compression algorithms. The training of such networks involves carefully trading off entropy of the latent representation against reconstruction quality. The term quality crucially depends on the observer of the images which, in the vast majority of literature, is assumed to be human. In this paper, we aim to go beyond this notion of compression quality and look at human visual perception and image classification simultaneously . To that end, we use a family of loss functions that allows to optimize deep image compression depending on the observer and to interpolate between human perceived visual quality and classification accuracy, enabling a more unified view on image compression. Our extensive experiments show that using perceptual loss functions to train a compression system preserves classification accuracy much better than traditional codecs such as BPG without requiring retraining of classifiers on compressed images. For example, compressing ImageNet to 0.25 bpp reduces Inception-ResNet classification accuracy by only 2%. At the same time, when using a human friendly loss function, the same compression system achieves competitive performance in terms of MS-SSIM. By combining these two objective functions, we show that there is a pronounced trade-off in compression quality between the human visual system and classification accuracy. Electronic supplementary material The online version of this chapter (10.1007/978-3-030-71278-5_10) contains supplementary material, which is available to authorized users.

Target-Dependent Scalable Image Compression Using a Reconfigurable Recurrent Neural Network

“…Elkholy et al studied the effects of compression and fusion techniques for image classification using various compression ratios [12]. Based on these findings, Weber et al designed an RNN based compression network that is trained by a variable loss function depending on the target [6]. The loss function they used was Loss = (1 − α) × M SSSIM + α × P erceptual, where scalability is provided by parameter α which interpolates between humancentric (MSSSIM) and task-centric (accuracy) metrics.…”

“…The baseline RNN-based compression network can control the bit-rate with the number of iterations, and its internal data flow can be easily reconfigured according to the compression target by altering its gate structure. [6] for target scalability. This method needs to maintain several pre-trained models for different targets in order to provide dynamic and various targets.…”

“…Content may change prior to final publication. To support the target scalability, the method in [6] needs several pre-trained models for the target interpolation parameter (Fig. 2 (a)).…”

“…The loss function used in this work is based on the one in [6], which is a convex combination of MSSSIM loss and VGG16 perceptual loss [13]. In Equation 1, α determines the tradeoff between the Loss M SSSIM and the Loss P erceptual , and β 1 and β 2 are hyper-parameters that balance these two loss functions from different spaces.…”

“…1 (top)). To this end, a recent study proposed a compression technique that trains a common neural network architecture that acts differently depending on the target [6] (Fig. 1 (middle)).…”