Depth Image Based View Synthesis: New Insights and Perspectives on Hole Generation and Filling

Zhu, Ce; Li, Shuai

doi:10.1109/tbc.2015.2475697

Cited by 73 publications

(24 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, the background depth value in a hole is estimated. As in our previous work [19], we have shown that the occluded background information can be used to determine whether the information around the hole belongs to background. After determining the background information around the hole, the (left or right) edge pixel of the hole which is adjacent to the background is then 6 assigned by the depth value of its most adjacent background pixel since the depth map is generally flat in the background regions.…”

Section: Selective Warping Of Complementary Viewsmentioning

confidence: 61%

See 1 more Smart Citation

Hole Filling With Multiple Reference Views in DIBR View Synthesis

Zhu

Sun

2018

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Selective Warping Of Complementary Viewsmentioning

confidence: 61%

“…There are currently some works on hole filling [11]- [19] available in the literature. Most of them employ image inpainting techniques [11]- [15] as they have been widely used to fill cracks or selected regions of an image.…”

Section: Introductionmentioning

confidence: 99%

Hole Filling With Multiple Reference Views in DIBR View Synthesis

Zhu

Sun

2018

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

“…In general, a virtual view can be obtained from the two nearest views or can be extrapolated from a single view. View-blending methods [4], [5] using multi-views have been studied in regard to the problem of filling holes. These methods have led to more promising results than single-view methods, but the hole-filling process took into account the interoperability and scalability between the views, thereby dramatically increasing the complexity as the number of reference views increased.…”

Section: Introductionmentioning

confidence: 99%

“…Inpainting methods can be classified as either structure-based or exemplar-based. It was reported that structure-based inpainting lost detailed information because of the diffusion of the structural elements [5]. Conversely, exemplar-based inpainting has been reported to solve the disocclusion problem, under the assumption that a known region and the neighboring unknown region of an image share a similar geometric structure [13].…”

Section: Introductionmentioning

confidence: 99%

Hole-Filling Algorithm with Spatio-Temporal Background Information for View Synthesis

Doan

Nguyen

Hong

2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThis paper presents a new hole-filling method that uses extrapolated spatio-temporal background information to obtain a synthesized free-view. A new background codebook for extracting reliable temporal background information is introduced. In addition, the paper addresses estimating spatial local background to distinguish background and foreground regions so that spatial background information can be extrapolated. Background holes are filled by combining spatial and temporal background information. Finally, exemplar-based inpainting is applied to fill in the remaining holes using a new priority function. The experimental results demonstrated that satisfactory synthesized views can be obtained using the proposed algorithm. key words: hole-filling, spatio-temporal background information, background codebook, spatial local background estimation, priority function

show abstract

“…While the new depth sensors are promising, the use of depth cameras is still limited by the low quality of the produced depth maps that have low resolution, noise, and depth missing in some areas. Therefore, effective post-processing and fusion techniques are needed to create high-quality depth maps for a truly 3D experience [3], [4]. Several methods have been proposed on the depth recovery from low-quality depth observations [3]- [15].…”

mentioning

confidence: 99%

Global Auto-Regressive Depth Recovery via Iterative Non-Local Filtering

Yang

Frossard

2019

IEEE Trans. on Broadcast.

View full text Add to dashboard Cite

Existing depth sensing techniques have many shortcomings in terms of resolution, completeness, and accuracy. The performance of 3-D broadcasting systems is therefore limited by the challenges of capturing high-resolution depth data. In this paper, we present a novel framework for obtaining high-quality depth images and multi-view depth videos from simple acquisition systems. We first propose a single depth image recovery algorithm based on auto-regressive (AR) correlations. A fixedpoint iteration algorithm under the global AR modeling is derived to efficiently solve the large-scale quadratic programming. Each iteration is equivalent to a nonlocal filtering process with a residue feedback. Then, we extend our framework to an ARbased multi-view depth video recovery framework, where each depth map is recovered from low-quality measurements with the help of the corresponding color image, depth maps from neighboring views, and depth maps of temporally adjacent frames. AR coefficients on nonlocal spatiotemporal neighborhoods in the algorithm are designed to improve the recovery performance. We further discuss the connections between our model and other methods like graph-based tools, and demonstrate that our algorithms enjoy the advantages of both global and local methods. Experimental results on both the Middleburry datasets and other captured datasets finally show that our method is able to improve the performances of depth images and multi-view depth videos recovery compared with state-of-the-art approaches. Index Terms-Depth recovery, multi-view depth video, autoregressive model, nonlocal correlation, iterative filtering. I. INTRODUCTION 3 D IMAGING applications have rapidly gained interest in recent years, as they promise to enhance the visual experience by extending the conventional 2D video to a more immersive experience. However, the technologies constituting 3D imaging systems are not fully mature yet to

show abstract

Depth Image Based View Synthesis: New Insights and Perspectives on Hole Generation and Filling

Cited by 73 publications

References 27 publications

Hole Filling With Multiple Reference Views in DIBR View Synthesis

Hole Filling With Multiple Reference Views in DIBR View Synthesis

Hole-Filling Algorithm with Spatio-Temporal Background Information for View Synthesis

Global Auto-Regressive Depth Recovery via Iterative Non-Local Filtering

Contact Info

Product

Resources

About