Adaptive Resolution Upconversion for Compressed Video Using Pixel Classification

Abstract

A novel adaptive resolution upconversion algorithm that is robust to compression artifacts is proposed. This method is based on classification of local image patterns using both structure information and activity measure to explicitly distinguish pixels into content or coding artifacts. The structure information is represented by adaptive dynamic-range coding and the activity measure is the combination of local entropy and dynamic range. For each pattern class, the weighting coefficients of upscaling are optimized by a least-mean-square (LMS) training technique, which trains on the combination of the original images and the compressed downsampled versions of the original images. Experimental results show that our proposed upconversion approach outperforms other classification-based upconversion and artifact reduction techniques in concatenation.

Context-based adaptive image resolution upconversion

“…If assuming Dirac downsampling, image resolution upconversion is commonly referred to as image interpolation. Existing image interpolation techniques fall into three categories: ͑1͒ image-independent linear interpolators, such as bilinear, cubic convolution 1 and cubic spline interpolators; 2 ͑2͒ adaptive linear interpolators, including some of the best performing interpolation techniques; [3][4][5][6] and ͑3͒ context-based interpolators [7][8][9] that perform the interpolation with an offline trained filter, according to the local LR pixel structures. The image-independent linear interpolators have the lowest computational complexity, hence they are favored for realtime applications.…”

An Overview and Performance Evaluation of Classification-Based Least Squares Trained Filters

“…The filter coefficients of video enhancement are usually heuristically optimized [1]- [5], which often involves tedious tuning and testing. Recently, classification-based least squares (LS) filters have been proposed for video enhancement applications including resolution upscaling [6], [7] and coding artifacts reduction [8], [9], which yield promising results. The experimental results in our previously published papers [7]- [9] show the superiority of the classification-based LS filters over other heuristically designed adaptive filters.…”

“…Recently, classification-based least squares (LS) filters have been proposed for video enhancement applications including resolution upscaling [6], [7] and coding artifacts reduction [8], [9], which yield promising results. The experimental results in our previously published papers [7]- [9] show the superiority of the classification-based LS filters over other heuristically designed adaptive filters. The main idea is that unclassified LS filters, i.e., the LS optimization is done on the image as a whole, may perform poorly on individual image regions, since a unique LS filter is designed for all pixels in an image.…”

“…In Section V, we conclude this paper and make some suggestions for implementation. The contributions of this paper, compared to previous works [7]- [9], include that we analyze the classification-based LS algorithm on different video enhancement scenarios and the most appropriate solutions are proposed for each application, which make the trained filters an excellent choice for picture quality improvement in modern display products. The previous works all attempt to tackle individual problems but have not put much effort on making this algorithm a generalized solution for video restoration, which is exactly the objective of this paper.…”