Visual attention guided bit allocation in video compression

Li, Zhicheng; Qin, Shiyin; Itti, Laurent

doi:10.1016/j.imavis.2010.07.001

Cited by 216 publications

(119 citation statements)

References 39 publications

Supporting

Mentioning

118

Contrasting

Unclassified

Order By: Relevance

“…In the former approach, called offline foveation, the image frames to be encoded are processed in such a way that the quality of the peripheral region is degraded (e.g., low-pass filtering), which are inputted to an existing encoder (e.g., [2], [25]). The latter approach applies different encoding parameters for the foveated and peripheral regions (e.g., [4]), or allocates more bits on the foveated region in a rate-control scheme (e.g., [26]). The foveated coding method used in this paper is based on the scheme existing in H.264/AVC, which allows us to use different quantization parameters (QPs) for different regions.…”

Section: B Foveated Video Codingmentioning

confidence: 99%

Subjective Quality Evaluation of Foveated Video Coding Using Audio-Visual Focus of Attention

Lee

Simone

Ebrahimi

2011

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Abstract-This paper presents a foveated coding method using audio-visual focus of attention and its evaluation through extensive subjective experiments on both standard-definition and high-definition sequences. Regarding a sound-emitting region as the location drawing the human attention, the method applies varying quality levels in an image frame according to the distance of a pixel to the identified sound source. Two experiments are presented to prove the efficiency of the method. Experiment 1 examines the validity and effectiveness of the method in comparison to the constant quality coding for high-quality conditions. In Experiment 2, the method is compared to the fixed bit rate coding for low quality conditions where coding artifacts are noticeable. The results demonstrate that the foveated coding method provides considerable coding gain without significant quality degradation, but uneven distributions of the coding artifacts (blockiness) by the method are often less preferred than the uniform distribution of the artifacts. Additional interesting findings are also discussed, such as content dependence of the performance of the method, the memory effect in multiple viewings, and the difference in the quality perception for frame size variations.Index Terms-Audio-visual focus of attention, content dependence, foveated coding, H.264/AVC, memory effect, quality of experience, subjective quality assessment.

show abstract

Section: B Foveated Video Codingmentioning

confidence: 99%

Subjective Quality Evaluation of Foveated Video Coding Using Audio-Visual Focus of Attention

Lee

Simone

Ebrahimi

2011

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

show abstract

“…o DC E is compared to an empirical threshold, th 1 . The result is a binary Smoothness Map (SM), with 'zero' value for non-homogeneous blocks and 'one' for those considered smooth:…”

Section: B Smoothness Detectionmentioning

confidence: 99%

“…This new paradigm is expected to provide a step forward in the achievement of better quality coding, especially for medium to low bit rates. The automatic estimation of the locations of the most prominent regions from a subjective point of view is, however, a challenging issue [1].…”

Section: Introductionmentioning

confidence: 99%

Low-complexity motion-based saliency map estimation for perceptual video coding

et al. 2011

View full text Add to dashboard Cite

Abstract-In this paper, a low-complexity motion-based saliency map estimation method for perceptual video coding is proposed. The method employs a camera motion compensated vector map computed by means of a hierarchical motion estimation (HME) procedure and a Restricted Affine Transformation (RAT)-based modeling of the camera motion. To allow for a computationally efficient solution, the number of layers of the HME has been restricted and the potential unreliable motion vectors due to homogeneous regions have been detected and specially managed by means of a smooth block detector. Special care has been taken of the smoothness of the resulting compensated camera motion vector map to avoid unpleasant artifacts in the perceptuallycoded sequence, by including a final post-processing based on morphological filtering. The proposed saliency map has been both visually and subjectively assessed showing quality improvements when used as a part of the H.264/AVC standard codec at medium-to-low bitrates.

show abstract

“…During the last few years another related track emerged where the goal is to segment out salient objects [10]- [20], instead of predicting some sparse eye-fixations. Both research tracks produce saliency maps that are useful for tasks such as video surveillance [21], compression [22], image manipulation [23], automatic image cropping [24], foreground detection [25], and coding [26], to name a few. However, the output of salient object detector techniques is more useful, when compared to eye fixation predictions, for higher level computer vision and pattern recognition …”

Section: Introductionmentioning

confidence: 99%

Probabilistic saliency estimation

Aytekin

Iosifidis

Gabbouj

2018

Pattern Recognition

View full text Add to dashboard Cite

Abstract-In this paper, we model the salient object detection problem under a probabilistic framework encoding the boundary connectivity saliency cue and smoothness constraints in an optimization problem. We show that this problem has a closed form global optimum which estimates the salient object. We further show that along with the probabilistic framework, the proposed method also enjoys a wide range of interpretations, i.e. graph cut, diffusion maps and one-class classification. With an analysis according to these interpretations, we also find that our proposed method provides approximations to the global optimum to another criterion that integrates local/global contrast and large area saliency cues. The proposed approach achieves mostly leading performance compared to the state-of-the-art algorithms over a large set of salient object detection datasets including around 17k images for several evaluation metrics. Furthermore, the computational complexity of the proposed method is favorable/comparable to many state-of-the-art techniques.

show abstract

Visual attention guided bit allocation in video compression

Cited by 216 publications

References 39 publications

Subjective Quality Evaluation of Foveated Video Coding Using Audio-Visual Focus of Attention

Subjective Quality Evaluation of Foveated Video Coding Using Audio-Visual Focus of Attention

Low-complexity motion-based saliency map estimation for perceptual video coding

Probabilistic saliency estimation

Contact Info

Product

Resources

About