Frame-Compatible Stereo 3D Services Using H.264/AVC and HEVC

Abstract

Stereoscopic 3D services are attracting considerable attention across various industries than never before and one of the major challenges is to introduce these services seamlessly while maintaining the backward compatibility of the existing 2D receivers. Also the increased amount of data for stereoscopic 3D needs to be efficiently compressed. In this paper an attempt has been made to compare the various options for the realization of frame compatible stereo 3D services along with the corresponding compression efficiency and backward compatibility issues. The first part of the work deals with realization of stereo 3D using Frame Packing Arrangement (FPA) Supplemental Enhancement Information (SEI). The problems associated with FPA SEI messages in H.264 including backward compatibility were discussed and the simulcast method of encoding base view and FPA SEI based stereo 3D video to mitigate the same has been outlined. In terms of HEVC specification, various contributions which have been proposed to resolve the issues that arose in H.264 were elaborated including addition of a flag to Video Usability Information / profile, tier and level syntax. The methods of encapsulating different views in independent entities (slices in H.264 and tiles in HEVC) have been proposed. The second part of the work deals with realization of stereo 3D using temporal scalability of scalable video coding (SVC-TS). The backward compatibility issues which arise in SVC-TS and the need to change the existing 2D infrastructure to accommodate this method have been discussed. Simulations have been setup to measure the compression efficiency of FPA SEI, simulcast method, SVC-TS & MVC(only in H.264) and simulation results shows that SVC-TS based stereoscopic 3D methods provides better compression efficiency when compared to FPA SEI based methods in both H.264 as well as in HEVC at the expense of increased computational complexity. Also in HEVC FPA SEI method, simulation results (as depicted in figure below for AKKO & KAIYO sequence) shows that encapsulating the two views in different "tiles" (with loop filters across tile boundaries disabled) have comparable R-D performance with "no-tiles" case, which will be useful in higher resolution video services such as 8Kx4K, 4Kx2K, where the two views could be decoded in parallel. 30 32 34 36 38 40 42 44 46 0 1000 2000 3000 Average PSNR(dB) bit rate (Kbps) HEVC_FPA_SS HEVC_FPA_TB HEVC_FPA_SS_TILES HEVC_FPA_TB_TILES HEVC_SVC_TS 2013 Data Compression Conference