Consistent Disparity Synthesis for Inter-View Prediction in Lightfield Compression

Abstract

For efficient compression of lightfields that involve many views, it has been found preferable to explicitly communicate disparity/depth information at only a small subset of the view locations. In this study, we focus solely on inter-view prediction, which is fundamental to multi-view imagery compression, and itself depends upon the synthesis of disparity at new view locations. Current HDCA standardization activities consider a framework known as WaSP, that hierarchically predicts views, independently synthesizing the required disparity maps at the reference views for each prediction step. A potentially better approach is to progressively construct a unified multi-layered base-model for consistent disparity synthesis across many views. This paper improves significantly upon an existing base-model approach, demonstrating superior performance to WaSP. More generally, the paper investigates the implications of texture warping and disparity synthesis methods.

JPEG Pleno Light Field Encoder with Mesh based View Warping

“…With the number of triangular cells being typically far smaller than the number of pixels, mesh-based representations also form a more compact description of the information conveyed by a depth map. In our prior efforts [9], mesh-based view warping was studied from a purely view synthesis perspective, omitting the steps of coding. In this work, we incorporate mesh-based view warping to the JPEG Pleno light field coding framework and report on R-D improvements.…”

“…Our earlier work [10] was conducted prior to the standardisation of the JPEG Pleno framework and therefore the R-D impact of incorporating mesh-based view warping to the standardized encoder was not previously investigated. Compared to previous efforts [9] [10], we provide improved dis-occlusion handling with a disciplined mesh augmentation strategy and report on benefits to JPEG Pleno encoding.…”