Towards efficient context-specific video coding based on gaze-tracking analysis

Abstract

This article discusses a framework for model-based, context-dependent video coding based on exploitation of characteristics of the human visual system. The system utilizes variable-quality coding based on priority maps which are created using mostly context-dependent rules. The technique is demonstrated through two case studies of specific video context, namely open signed content and football sequences. Eye-tracking analysis is employed for identifying the characteristics of each context, which are subsequently exploited for coding purposes, either directly or through a gaze prediction model. The framework is shown to achieve a considerable improvement in coding efficiency.

Gaze location prediction in broadcast football video

“…Then all five d to output z s , for moving es [11] find that: in global st football videos, the ball portant objects that would as in close up shots most head and text on the shirt. the salient object in global ndition of close-up shots examined in [9].…”

Objective Assessment of Region of Interest-Aware Adaptive Multimedia Streaming Quality

“…In more recent work, Agrafiotis et al [27] present a framework for model-based, context-dependent video coding, based on exploitation of characteristics of the human visual system. The system utilizes variable-quality coding, based on priority maps which are created using mostly contextdependent rules.…”

A Multicue Bayesian State Estimator for Gaze Prediction in Open Signed Video

“…(1) (2) For the 7000 ground truthed frames, and five study participants, values of and were obtained. This implies that 95% of the time the observer is looking at the signer, signing is taking place, and that for 82% of the time signing is taking place the observers are concentrating on the signer.…”

“…Different video contexts have different gaze patterns associated with them e.g., football and open sign language [2]. Due to the omission of any top-down processes, saliency has been shown to be inadequate for such video contexts (including open sign language [11]).…”