Redundant representation for network video streaming using reconstructed P-frames and SP-frames

Abstract

For low-delay streaming of pre-encoded video over lossy networks, fast recovery from decoding errors typically involves use of frequent intra-coded frames, which incurs high bandwidth cost. In this paper, we present a redundant representation of video using a bandwidth-efficient but non-resilient main bitstream, and an additional auxiliary bitstream dedicated to recovery from losses in the main bitstream. In particular, we insert primary SP-frames periodically into the main bitstream, and encode corresponding secondary SP-frames and reconstructed P-frames in the auxiliary stream. When a frame loss occurs, a reconstructed P-frame is first sent to re-synchronize decoder back to normal motion compensation loop, then a secondary SP-frame corresponding to the location of the next pre-inserted primary SP-frame in the main stream is sent thereafter, eliminating coding drift. Results show that proposed method out-performs non-redundant representation of I-frame insertion by up to 11 frames in recovery time, and out-performed redundant representation of only reconstructed P-frames by up to 2.2dB in average PSNR.