Error Concealment in Encoded Video Streams

Abstract

When transmitting compressed video over a data network, one has to deal with how channel errors affect the decoding process. This is particularly problematic with data loss or erasures. In this paper we describe techniques to address this problem in the context of networks where channel errors or congestion can result in the loss of entire macroblocks when MPEG video is transmitted. We describe spatial and temporal techniques for the recovery of lost macroblocks. In particular, we develop estimation techniques for the reconstruction of missing macroblocks using a Markov Random Field model. We show that the widely used heuristic motion compensated error concealment technique based on averaging motion vectors is a special case of our estimation technique. We further describe a technique that can be implemented in real-time.

<title>Streaming video and rate-scalable compression: What are the challenges for watermarking?</title>

“…The error resilience features in MPEG-4 [30] include reversible variable-length codes, "video packets" (similar to H.263+ slices) and data partitioning for resynchronization, and the ability to add redundant copies of header information (to avoid dropping entire frames when headers are lost or corrupted.) Finally, error concealment [25,31] can be used to minimize the visual impact arising from data loss when the video is displayed to the user by predicting the contents of missing or corrupted areas in the video.…”

Motion field estimation by vector rational interpolation for error concealment purposes

“…When motion compensated coding has been employed, which is the case with the MPEG-2 compression algorithm [1], errors observed in a frame propagate to subsequent in coding order frames leading, thus, to even worse visual quality. Error concealment methods have emerged to deal with such problems at the decoder [2,3,4,5]. Many of these methods attempt to estimate lost motion information for predictively coded frames (P-or B-frames) and conceal lost blocks by motion compensated temporal replacement [2,3,4,5].…”

“…Error concealment methods have emerged to deal with such problems at the decoder [2,3,4,5]. Many of these methods attempt to estimate lost motion information for predictively coded frames (P-or B-frames) and conceal lost blocks by motion compensated temporal replacement [2,3,4,5]. Such methods prove to be fast and attain satisfactory concealment results.…”

“…Among the motion field estimation error concealment (EC) methods, one can distinguish the zero motion EC (ZM EC), the motion compensated EC (MC EC), the boundary matching algorithm EC (BMA EC) [2], the motion vector estimation by boundary optimizing EC (MVE-BO EC) [5] and the forward-backward block matching EC (F-B BM EC) [4]. The ZM EC sets the lost motion vectors to zero.…”

Complexity-regularized image restoration

“…For instance, blocks of image data may be lost due to packet loss in ATM networks over which the image is transmitted [27]. Assuming the receiver knows which packets have been lost, the degradation model mapping the original x to the received y is again deterministic and many-to-one.…”