Video Compression for Lossy Packet Networks With Mode Switching and a Dual-Frame Buffer

Abstract

Video codecs that use motion compensation benefit greatly from the development of algorithms for near-optimal intra/inter mode switching within a rate-distortion framework. A separate development has involved the use of multiple-frame prediction, in which more than one past reference frame is available for motion estimation. In this paper, we show that using a dual-frame buffer (one short-term frame and one long-term frame available for prediction) together with intra/inter mode switching improves the compression performance of the coder. We improve the mode-switching algorithm with the use of half-pel motion vectors. In addition, we investigate the effect of feedback in making more informed and effective mode-switching decisions. Feedback information is used to limit drift errors due to packet losses by synchronizing the long-term frame buffers of both the encoder and the decoder.

Error Resilient Video Coding Using B Pictures in H.264

“…Theoretical Error Ratios calculated by (22) are also plotted, which are very close to those obtained from the simulation data. From (22) we can see that the Error Ratios of Type2 and Type3 are close for a small h 1 (i.e., h 1 = 0.125), while the Error Ratios of Type1 and Type3 are close for a large h 1 (i.e., h 1 = 0.875). In addition, increasing h 1 can make the Error Ratio larger.…”

“…This is analogous to the use of a long-term reference frame in multiple-referenceframe video coding, which has been shown to have the ability to improve both the compression efficiency and error robustness of the encoded videos [20]- [22]. The authors in [20] consider a rate-constrained motion-estimation framework to select the temporal delay parameter (reference frame) and motion vectors.…”

“…The authors use Markov chain analyses to derive a rule to randomize the selection of the reference frame and thus improve the robustness of the codec. In [22], a dualframe buffer, where one short-term frame and one long-term frame are maintained as reference frames, is used together with an INTRA/INTER mode switching algorithm to improve the encoder's compression efficiency and the robustness of the compressed bitstream. In all these multiple-referenceframe coding algorithms only one of the references (single hypothesis) will be used to predict a MB, whereas a linear combination of multiple signals (hypotheses) will be used to predict a MB in MHMCP.…”

Optimizing Motion Compensated Prediction for Error Resilient Video Coding

“…Conventional ME techniques improve performance in terms of source coding efficiency without explicit accounting for error robustness considerations in video streaming applications, e.g., various rate-constrained ME techniques [12]- [14], or RD-optimized ME [7], [15], [16]. Since motion vectors (MVs) are directly and critically involved in the error propagation mechanism encountered in transmission over lossy packet networks, ME may have a considerable impact on error resilience.…”

“…In this paper, we focus on the scenario of live video streaming (e.g., video conferencing/telephony) where ED-RDO can further be applied to optimize the source coding modules and options. For example, at the macro-block (MB) level, optimization may involve various coding modes or parameters including: intra/intramode [4], [5], quantization step size [4], prediction reference frame [6], [7], coding modes associated with layered coding [8], [9] or multiple description coding [10], [11], etc. Fig.…”