Graph-Based Representation for Multiview Image Geometry

Abstract

Abstract-In this paper, we propose a new geometry representation method for multiview image sets. Our approach relies on graphs to describe the multiview geometry information in a compact and controllable way. The links of the graph connect pixels in different images and describe the proximity between pixels in 3D space. These connections are dependent on the geometry of the scene and provide the right amount of information that is necessary for coding and reconstructing multiple views. Our multiview image representation is very compact and adapts the transmitted geometry information as a function of the complexity of the prediction performed at the decoder side. To achieve this, our graph-based representation (GBR) carefully selects the amount of geometry information needed before coding. This is in contrast with depth coding, which directly compresses with losses the original geometry signal, thus making it difficult to quantify the impact of coding errors on geometry-based interpolation. We present the principles of this GBR and we build an efficient coding algorithm to represent it. We compare our GBR approach to classical depth compression methods and compare their respective view synthesis qualities as a function of the compactness of the geometry description. We show that GBR can achieve significant gains in geometry coding rate over depth-based schemes operating at similar quality. Experimental results demonstrate the potential of this new representation.Index Terms-Multiview image coding, 3D representation, view prediction, graph-based representation.

Optimal Representations for Adaptive Streaming in Interactive Multiview Video Systems

“…The main open challenges on interactive multiview systems are: how to efficiently encode distributed sources, how to efficiently synthesize views, and how to efficiently deliver information to users in different applications. Coding efficiency in MV systems have been studied in the case of no communication among cameras, (e.g., distributed video coding [14]), or in the case of joint source coding, (e.g., HEVC for MV [15], graph-based coding and representation [16], [17], or coding for interactive users [18], [19]). To improve the process of view synthesis, several works have been focused mainly on improving the DIBR technique, [20]- [23].…”

Depth map compression via 3D region-based representation

“…In [20], a geometrical representation is introduced to describe the multiview information with a graph. Starting from an initial view, inter-view redundancy is avoided by adding new graph nodes only if new information appears in the subsequent views.…”

Graph-based light fields representation and coding using geometry information

“…GBR has been proposed for describing the geometry of multi-view images, first for horizontally aligned cameras [9] and more recently for complex camera configurations [10]. Here, we consider GBR to represent LF using 3D geometry information.…”