Learning a Convolutional Neural Network for Image Compact-Resolution

Li, Yue; Liu, Dong; Li, Houqiang; Li, Li; Zhu, Li; Wu, Feng

doi:10.1109/tip.2018.2872876

Cited by 128 publications

(113 citation statements)

References 40 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…The task aware downscaled image is obtained by jointly training the encoder and decoder to maximize the SR performance. Similar to the framework presented by [21], Li et al [4] proposed a convolutional neural network for image compact resolution named CNN-CR, which is composed of a CNN to estimate the LR image and a learned or specified upscaling model to reconstruct the HR image. Although the above three mentioned methods did not employ any ground-truth LR image, the generative nature of the encoder like networks implicitly require additional information to constrain the output to be a valid image whose content resembles the HR image.…”

Section: Task Specific Image Downscalingmentioning

confidence: 99%

“…We design the proposed content adaptive image downscaling model that is trained using the unsupervised strategy. Unlike work presented in [20,21,4] which synthesizes the downscaled image by combining latent representations of the HR image extracted by the CNN, and proper constraints are required to make sure that the result is a meaningful image. We propose to obtain the downscaled image using the idea of resampling the HR image, which effectively makes the downscaled result look like the original HR image without any constraints.…”

Section: Content Adaptive Image Downscaling Resamplermentioning

confidence: 99%

“…Thus, it is desirable to develop efficient image downscaling and upscaling method to make such application more practical and resources saving by only generating, storing and transmitting a single downscaled version for preview and upscaling it to high resolution when details are going to be viewed. Besides, the pre-downscaling and post-upscaling operation also helps to save storage and bandwidth for image or video compression and communication [1,2,3,4].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learned Image Downscaling for Upscaling Using Content Adaptive Resampler

Sun

Chen

2020

IEEE Trans. on Image Process.

146

View full text Add to dashboard Cite

Deep convolutional neural network based image super-resolution (SR) models have shown superior performance in recovering the underlying high resolution (HR) images from low resolution (LR) images obtained from the predefined downscaling methods. In this paper we propose a learned image downscaling method based on content adaptive resampler (CAR) with consideration on the upscaling process. The proposed resampler network generates content adaptive image resampling kernels that are applied to the original HR input to generate pixels on the downscaled image. Moreover, a differentiable upscaling (SR) module is employed to upscale the LR result into its underlying HR counterpart. By back-propagating the reconstruction error down to the original HR input across the entire framework to adjust model parameters, the proposed framework achieves a new state-of-the-art SR performance through upscaling guided image resamplers which adaptively preserve detailed information that is essential to the upscaling. Experimental results indicate that the quality of the generated LR image is comparable to that of the traditional interpolation based method, but the significant SR performance gain is achieved by deep SR models trained jointly with the CAR model. The code is publicly available on: URL https://github.com/sunwj/CAR.

show abstract

Section: Task Specific Image Downscalingmentioning

confidence: 99%

Section: Content Adaptive Image Downscaling Resamplermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learned Image Downscaling for Upscaling Using Content Adaptive Resampler

Sun

Chen

2020

IEEE Trans. on Image Process.

146

View full text Add to dashboard Cite

show abstract

“…For example, this can be seen in previous work [8], [53], [55], where input volumes are reduced by distilling input sequences to their most useful elements before relaying to remote servers for semantic analysis. Other work [22], [52] mainly focused on task-specific mappings of inputs onto lower-dimensional space before training with more dataefficient models, and recent advances in domain adaptation and transfer learning [26], [39], [48] can also be used to learn compressed codes tuned to particular models. However, for any specified source distribution, domain adaptation [26], [39], [48] and other proposals mentioned above [8], [53], [55] equally compact all sampled inputs to fixed length codes, and varying degrees of entropy among input examples are ignored.…”

Section: Related Workmentioning

confidence: 99%

Biased Mixtures of Experts: Enabling Computer Vision Inference Under Data Transfer Limitations

Abbas

Andreopoulos

2020

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

We propose a novel mixture-of-experts class to optimize computer vision models in accordance with data transfer limitations at test time. Our approach postulates that the minimum acceptable amount of data allowing for highly-accurate results can vary for different input space partitions. Therefore, we consider mixtures where experts require different amounts of data, and train a sparse gating function to divide the input space for each expert. By appropriate hyperparameter selection, our approach is able to bias mixtures of experts towards selecting specific experts over others. In this way, we show that the data transfer optimization between visual sensing and processing can be solved as a convex optimization problem. To demonstrate the relation between data availability and performance, we evaluate biased mixtures on a range of mainstream computer vision problems, namely: (i) single shot detection, (ii) image super resolution, and (iii) realtime video action classification. For all cases, and when experts constitute modified baselines to meet different limits on allowed data utility, biased mixtures significantly outperform previous work optimized to meet the same constraints on available data.

show abstract

“…However, this approach is quite designed to meet very specific requirement of particular application. Most recently, the work carried out by Li et al [37] has addressed this problem using convolution neural network. The main idea of this paper is to showcase that the recent video compression standard of HEVC can be out-performed where the presented technique is about obtaining a high-resolution frame from lower version of it.…”

Section: B Typical Machine Learning Approachmentioning

confidence: 99%

Insights on Video Compression Strategies using Machine Learning

Rao

2019

IJEAT

View full text Add to dashboard Cite

With the rising advancement of the multimedia technology, video compression is becoming a challenging problem. Although, there is availability of various standard compression algorithms, yet robust compression performance is yet to be seen in existing compression techniques. This paper also highlights that machine learning plays a significant contributory role in improving the performance of the video compression. Therefore, this manuscript offers a technical insight about the performance of existing video compression technique using machine learning approach. The contribution of this paper is its findings which states that machine learning approach do have significant advantage but the advantageous features are limited by the inherent and unsolved research problem. The core findings of this paper are basically to highlight the strength and limitations of existing methods as well as to highlight the research gap in terms of open-end research problems which requires immediate attention.

show abstract

Learning a Convolutional Neural Network for Image Compact-Resolution

Cited by 128 publications

References 40 publications

Learned Image Downscaling for Upscaling Using Content Adaptive Resampler

Learned Image Downscaling for Upscaling Using Content Adaptive Resampler

Biased Mixtures of Experts: Enabling Computer Vision Inference Under Data Transfer Limitations

Insights on Video Compression Strategies using Machine Learning

Contact Info

Product

Resources

About