Scalable Coding of Plenoptic Images by Using a Sparse Set and Disparities

Abstract

This material is published in the open archive of Mid Sweden University DIVA http://miun.diva-portal.org to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders.All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder. Abstract-One of the light field capturing techniques is the focused plenoptic capturing. By placing a microlens array in front of the photosensor, the focused plenoptic cameras capture both spatial and angular information of a scene in each microlens image and across microlens images. The capturing results in significant amount of redundant information, and the captured image is usually of a large resolution. A coding scheme that removes the redundancy before coding can be of advantage for efficient compression, transmission and rendering. In this paper, we propose a lossy coding scheme to efficiently represent plenoptic images. The format contains a sparse image set and its associated disparities. The reconstruction is performed by disparity-based interpolation and inpainting, and the reconstructed image is later employed as a prediction reference for the coding of the full plenoptic image. As an outcome of the representation, the proposed scheme inherits a scalable structure with three layers. The results show that plenoptic images are compressed efficiently with over 60 percent bit rate reduction compared to HEVC intra, and with over 20 percent compared to HEVC block copying mode.article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.

Light Field Compression With Disparity-Guided Sparse Coding Based on Structural Key Views

“…Since each pixel in a MAI represents a separate viewing angle, therefore the correlation within each MAI actually represents angular correlation. Recent LF compression methods proposed by Li et al [11] and Conti et al [12], [13] both work on MLIs. Li et al [11] attempted to remove the angular correlation via disparity guided view interpolation based on a sparse set of sub-sampled angular examples.…”

“…Recent LF compression methods proposed by Li et al [11] and Conti et al [12], [13] both work on MLIs. Li et al [11] attempted to remove the angular correlation via disparity guided view interpolation based on a sparse set of sub-sampled angular examples. Conti et al [12], [13] proposed the self-similarity concept to compensate repeated MAI patterns.…”

Light Field Compression With Homography-Based Low-Rank Approximation

“…While the elemental images (EI) in the focused 2.0 plenoptic cameras can be seen as a cropped multi-view image from one viewing angle, the EI produced by the unfocused 1.0 cameras capture angular information of one point in space. A scalable extension of HEVC-based scheme is also proposed in [45] where a sparse set of micro-lens images (also called elemental images) is encoded in a base layer. The other elemental images are reconstructed at the decoder using disparity-based interpolation and inpainting.…”

Light field compression using depth image based view synthesis

“…Approaches based on HEVC mostly focus on the introduction of dedicated prediction modes. A scalable extension of HEVC-based scheme is proposed in [12] where a sparse set of micro-lens images (also called elemental images) is encoded in a base layer. The other elemental images are reconstructed at the decoder using disparity-based interpolation and inpainting.…”