Scene-library-based video coding scheme exploiting long-term temporal correlation

Abstract

, "Scene-library-based video coding scheme exploiting long-term temporal correlation," J. Electron. Imaging 26(4), 043026 (2017), doi: 10.1117/1.JEI.26.4.043026. Abstract. In movies and TV shows, it is common that several scenes repeat alternately. These videos are characterized with the long-term temporal correlation, which can be exploited to improve video coding efficiency. However, in applications supporting random access (RA), a video is typically divided into a number of RA segments (RASs) by RA points (RAPs), and different RASs are coded independently. In such a way, the long-term temporal correlation among RASs with similar scenes cannot be used. We present a scene-library-based video coding scheme for the coding of videos with repeated scenes. First, a compact scene library is built by clustering similar scenes and extracting representative frames in encoding video. Then, the video is coded using a layered scene-library-based coding structure, in which the library frames serve as long-term reference frames. The scene library is not cleared by RAPs so that the long-term temporal correlation between RASs from similar scenes can be exploited. Furthermore, the RAP frames are coded as interframes by only referencing library frames so as to improve coding efficiency while maintaining RA property. Experimental results show that the coding scheme can achieve significant coding gain over state-of-the-art methods. © The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Apron surveillance video coding based on compositing virtual reference frame with object library

“…[10] proposed to extract the common scene frame which stands for the stable content in a scene, and the original I frame in the same scene is predicted by the selective common scene frame as a long‐term reference. In [11], in order to make full use of the common scene frame, besides the I frame, the P frame can also refer to library frames for inter‐prediction. Focusing on obtaining the optimal long‐term reference frame, Jubran et al.…”

“…Considering that several scenes repeat alternately in movies and TV shows, Zuo et al [10] proposed to extract the common scene frame which stands for the stable content in a scene, and the original I frame in the same scene is predicted by the selective common scene frame as a long-term reference. In [11], in order to make full use of the common scene frame, besides the I frame, the P frame can also refer to library frames for inter-prediction. Focusing on obtaining the optimal long-term reference frame, Jubran et al [12] use the structural similarity of pictures to select the sequence-level long-term reference and obtain significant rate gains.…”

“…To obtain the optimal estimation for (t x , t y ), the fine-grained feature, scale-invariant feature transform (SIFT) key points [39], are chosen to align the retrieved reference object and to-be-code object. The multiple matched FIGURE 11 The comparison of the number of objects between the object library and I frames.…”