No abstract

No abstract

We introduce a general method of performing Residual Network inference and learning in the JPEG transform domain that allows the network to consume compressed images as input. Our formulation leverages the linearity of the JPEG transform to redefine convolution and batch normalization with a tune-able numerical approximation for ReLu. The result is mathematically equivalent to the spatial domain network up to the ReLu approximation accuracy. A formulation for image classification and a model conversion algorithm for spatial domain networks are given as examples of the method. We show skipping the costly decompression step allows for faster processing of images with little to no penalty in the network accuracy.1. The general method for expressing convolutional networks in the JPEG domain 2. Concrete formulation for residual blocks to perform classification 3. A model conversion algorithm to apply pretrained spatial domain networks to JPEG images 4. Approximated Spatial Masking: the first general technique for application of piecewise linear functions in the transform domain By skipping the decompression step and by operating on the compressed format, we show a notable increase in speed for testing and a marginal speed for training. Prior WorkWe review prior work separated into three categories: compressed domain operations, machine learning in the compressed domain, and deep learning in the compressed domain. Compressed Domain OperationsThe expression of common operations in the compressed domain was an extremely active area of study in the late 80s and early 90s, motivated by the lack of computing power to 1