Perceptually optimised coding of open signed video based on gaze prediction

A gaze prediction technique for open signed video content using a track before detect algorithm

“…Previous work [2] has shown that in open signed material there are two especially important regions -the signer's face and, to a lesser extent, the inset showing the original video.…”

“…Introducing the cues demonstrated in [2] could also see an improvement in performance. The gaze predictions can be used for a variety of applications, including variable quality coding and error protection.…”

“…Open sign language is an excellent candidate for perceptual video coding due to the well-defined nature of the gaze patterns, as shown in [1], [2], [3] and [4]. Previous work has been done on analysing the gaze patterns of sign language in various forms, including videoconferencing [4] and open sign video [2], and both went on to code video with perceptually varying quality. It was shown that within the sign language context it is possible to make coding gains without loss of intelligibility or perceived quality.…”

“…are combined into a saliency map, and this is then used to plot predicted gaze patterns. Due to the bottom-up nature of this scheme it has been shown to be inadequate for many specified contexts, sign language included [2].…”

A Multicue Bayesian State Estimator for Gaze Prediction in Open Signed Video

“…Due to the omission of any top-down processes, saliency has been shown to be inadequate for such video contexts (including open sign language [11]). Other techniques [8] introduce the idea of a top-down approach, but rely instead on categorizing objects within an image and then associating predetermined probabilities of eye-fixation with each object.…”