Unconstrained motion compensated temporal filtering (UMCTF) for efficient and flexible interframe wavelet video coding

Abstract

We introduce an efficient and flexible framework for temporal filtering in wavelet-based scalable video codecs called unconstrained motion compensated temporal filtering (UMCTF). UMCTF allows for the use of different filters and temporal decomposition structures through a set of controlling parameters that may be easily modified during the coding process, at different granularities and levels. The proposed framework enables the adaptation of the coding process to the video content, network and end-device characteristics, allows for enhanced scalability, content-adaptivity and reduced delay, while improving the coding efficiency as compared to state-of-the-art motion-compensated wavelet video coders. Additionally, a mechanism for the control of the distortion variation in video coding based on UMCTF employing only the predict step is proposed. The control mechanism is formulated by expressing the distortion in an arbitrary decoded frame, at any temporal level in the pyramid, as a function of the distortions in the reference frames at the same temporal level. All the different scenarios proposed in the paper are experimentally validated through a coding scheme that incorporates advanced features (such as rate-distortion optimized variable block-size multihypothesis prediction and overlapped block motion compensation). Experiments are carried out to determine the relative efficiency of different UMCTF instantiations, as well as to compare against the current state-of-the-art in video coding. r

Embedded Multiple Description Coding of Video

“…In a first step, motion-compensated temporal filtering of the input is employed in order to remove the temporal correlation among different frames: in essence, MCTF corresponds to a discrete wavelet transform (DWT) performed along the motion trajectories. The temporal transform is implemented via lifting [23], as illustrated in the simple example of Haar MCTF [6], [22] shown in Fig. 1.…”

“…Finally, we indicate the subband of a frame as . With these notations, (5) compacts into (6), shown at the bottom of the next page. The above formulation of the overall distortion in the reconstructed sequence is particularly useful because the proposed system quantizes each subband and motion field independently, and each contribution to the final distortion is individually evaluated at encoding time, as explained in the following section.…”

“…Given (9), we can formulate the expected distortion at the decoder side assuming packet erasures. Each time transmission is performed, a different approximation of is reconstructed depending on the loss pattern, as the decoder processes (6) all the packets which do not depend on missing data and discards the others. Then, the average distortion at the decoding site can be estimated as (10) where the set of vectors is Equation (10) averages the distortion yielded by decoding only the initial packets over all occurrences .…”

3-D video coding with redundant-wavelet multihypothesis

“…The salient aspect of this coder lies in that we employ multihypothesis motion compensation (MHMC) within the MCTF to combat the uncertainty inherent in estimating motion trajectories for MCTF, thereby achieving rate-distortion performance significantly superior to the usual single-hypothesis MCTF approach. Although multihypothesis has been used in conjunction with MCTF before (e.g., [9]- [11], [15] propose both spatially and temporally diverse multihypothesis MCTF predictions), in our proposed system, we employ a new class of MHMC-phase-diversity multihypothesis [17], [18]. Specifically, phase-diversity MHMC is implemented by deploying MCTF in the domain of a spatially redundant wavelet transform such that multiple hypothesis temporal filterings are combined implicitly in the form of an inverse transform.…”

Multiple Description Coding versus Transport Layer FEC for Resilient Video Transmission

“…This filtering is performed in the direction of motion. Assuming HAAR wavelets, MCTF can be written using the lifting formulation [7] as follows:…”