Semantic Perceptual Image Compression Using Deep Convolution Networks

Abstract

It has long been considered a significant problem to improve the visual quality of lossy image and video compression. Recent advances in computing power together with the availability of large training data sets has increased interest in the application of deep learning cnns to address image recognition and image processing tasks. Here, we present a powerful cnn tailored to the specific task of semantic image understanding to achieve higher visual quality in lossy compression. A modest increase in complexity is incorporated to the encoder which allows a standard, off-the-shelf jpeg decoder to be used. While jpeg encoding may be optimized for generic images, the process is ultimately unaware of the specific content of the image to be compressed. Our technique makes jpeg content-aware by designing and training a model to identify multiple semantic regions in a given image. Unlike object detection techniques, our model does not require labeling of object positions and is able to identify objects in a single pass. We present a new cnn architecture directed specifically to image compression, which generates a map that highlights semantically-salient regions so that they can be encoded at higher quality as compared to background regions. By adding a complete set of features for every class, and then taking a threshold over the sum of all feature activations, we generate a map that highlights semantically-salient regions so that they can be encoded at a better quality compared to background regions. Experiments are presented on the Kodak PhotoCD dataset and the MIT Saliency Benchmark dataset, in which our algorithm achieves higher visual quality for the same compressed size. 1 1 Code for the model is available at https

Protecting JPEG Images Against Adversarial Attacks

“…Table 1 The JPEG quantization level was chosen to achieve the highest accuracy. For MSROI, we applied the technique described in [17] and set the quantization level to keep the final image quality as close as possible to the JPEG version. The results are shown in Figure 2…”

“…Saliency detection techniques can solve these issues, but such techniques are limited in their ability to detect multiple objects, and the identified salient region may only contain a limited subset of the objects in the image. To address these shortcomings, we utilize MSROI [17], a CNN designed to retrieve all salient regions and provide a soft boundary over the image.…”

CocoNet: A Deep Neural Network for Mapping Pixel Coordinates to Color Values

“…To the best of our knowledge, there are no previous scientific works that propose to learn a mapping of the pixel coordinates to the corresponding pixel color values using neural networks. However, there are numerous neural models that learn a mapping from image pixels to a set of classes [10,16,30,33] or from pixels to pixels [1,2,[4][5][6]12,14,18,20,21,24,28,29,35,[39][40][41][42]. The neural models that map pixels to pixels are usually applied on tasks such as image compression [1,2,4,6,21,24,35], image denoising and restoration [20,39,42], image super-resolution [5,14,18,20,28,29,39,40], image completion [12,41] and image generation [11,36].…”

“…Dumas et al [6] address image compression using sparse representations, by proposing a stochastic winner-takes-all auto-encoder in which image patches compete with one another when their sparse representation is computed. Prakash et al [24] design a technique that makes JPEG content-aware by training a deep CNN model to generate a map that highlights semantically-salient regions so that they can be encoded at higher quality as compared to background regions. Toderici et al [35] present several recurrent neural network (RNN) architectures that provide variable compression rates during deployment without requiring retraining.…”

An FPGA Implementation of a Convolutional Auto-Encoder

“…David E. Rumelhart first proposed the concept of an auto-encoder [6] and employed it to process data with large dimensions, which promoted the development of neural networks. In 2006, Hinton et al [7] improved the original shallow auto-encoder and proposed the concept of a deep learning neural network as well as its training strategy, which can be used in the signal processing field for applications such as feature extraction [8], image compression [9][10][11], classification [12,13], image denoising [14], prediction [15], and so on. Wang et al [16] proposed a rapid 3D feature learning method named a convolutional auto-encoder extreme learning machine (CAE-ELM), and the features extracted were superior to other previous deep learning methods.…”