Multiview video coding for 3D telepresence

Abstract

Telepresence refers to a human/machine system in which the human operator feels physically present at a remote site. Telepresence systems have high potential in various applications of consumer entertainment.To achieve telepresence, stereoscopic and multiview video that can give the viewers the 3D feeling are required. The major obstacle for telepresence systems is the extremely large amount of data associated with it. Efficient compression of multiview video sequences has become an integral and important part of any practical telepresence system. To code the multiview video sequences efficiently, both the view redundancy among the sequences and the temporal redundancy inside each sequence need to be exploited and reduced, which can be achieved using disparity and motion compensation, respectively.The coding efficiency can be further improved by jointly estimating and optimizing motion and disparity fields.In this thesis, we develope several novel algorithms and two complete encoders for multiview video coding (MVC). The major contributions can be summarized in three aspects. In particular, we first propose an edge-preserving regularization scheme to calculate either 1D disparity fields or 2D motion fields. After confirming its performance by comparing with the existing algorithms, the separate regularization scheme is extended to a joint estimation scheme that calculates two disparity fields and two motion fields for two successive image pairs simultaneously.Secondly, we develop an MPEG-4 compatible multiview video encoder which integrates with the joint disparity and motion estimation scheme. Besides, various aspects of the encoder are investigated, including a comparative study of several view-level frame prediction structures and the rate control algorithm for multiview video coding.Thirdly, we propose a framework of scalable MVC using 4D wavelet. The waveletiii